Aug 03 16:02:13.167966 ip-10-0-138-110 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Aug 03 16:02:13.167976 ip-10-0-138-110 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Aug 03 16:02:13.167986 ip-10-0-138-110 systemd[1]: kubelet.service: Failed with result 'resources'.
Aug 03 16:02:13.168327 ip-10-0-138-110 systemd[1]: Failed to start Kubernetes Kubelet.
Aug 03 16:02:23.336816 ip-10-0-138-110 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Aug 03 16:02:23.336834 ip-10-0-138-110 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2ca4b652301d409bb19ca7ded502380b --
Aug 03 16:04:17.626036 ip-10-0-138-110 systemd[1]: Starting Kubernetes Kubelet...
Aug 03 16:04:18.005990 ip-10-0-138-110 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 16:04:18.005990 ip-10-0-138-110 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 03 16:04:18.005990 ip-10-0-138-110 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 16:04:18.005990 ip-10-0-138-110 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 03 16:04:18.005990 ip-10-0-138-110 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 16:04:18.006655 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.006005    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 03 16:04:18.011828 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011800    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:04:18.011828 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011819    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:04:18.011828 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011826    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:04:18.011828 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011830    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:04:18.011828 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011834    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011838    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011843    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011847    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011850    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011854    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011858    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011861    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011865    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011868    2564 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011873    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011877    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011881    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011884    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011888    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011891    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011896    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011900    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011904    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011908    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:04:18.012124 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011911    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011929    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011932    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011937    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011941    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011945    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011950    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011954    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011958    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011962    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011969    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011975    2564 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011980    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011984    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011990    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.011995    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012001    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012005    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012010    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012014    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:04:18.012954 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012018    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012023    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012027    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012031    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012036    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012041    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012049    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012055    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012060    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012065    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012069    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012074    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012078    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012082    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012087    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012091    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012096    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012101    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012107    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:04:18.013799 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012113    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012119    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012123    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012127    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012131    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012135    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012141    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012146    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012150    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012155    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012159    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012166    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012170    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012175    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012179    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012183    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012187    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012192    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012196    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012201    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:04:18.014649 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012206    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012211    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012215    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012219    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012223    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012227    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012232    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012236    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012241    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012245    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012249    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012253    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012257    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012261    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012265    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.012269    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013553    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013564    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013569    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:04:18.015651 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013584    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013590    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013595    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013600    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013605    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013609    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013614    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013618    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013623    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013628    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013632    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013637    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013643    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013649    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013655    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013660    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013665    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013669    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013673    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013678    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:04:18.016482 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013682    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013686    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013691    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013696    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013700    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013705    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013709    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013714    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013718    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013723    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013727    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013731    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013754    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013759    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013763    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013768    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013772    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013776    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013780    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013784    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:04:18.017168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013788    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013793    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013796    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013800    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013804    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013808    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013812    2564 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013817    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013822    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013826    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013830    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013835    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013838    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013843    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013848    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013852    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013857    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013861    2564 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013865    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013870    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:04:18.017868 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013874    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013878    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013882    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013886    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013891    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013895    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013899    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013903    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013907    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013912    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013916    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013920    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013925    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013929    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013936    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013941    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013945    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013950    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013955    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013959    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:04:18.018459 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013964    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013969    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013973    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013977    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013981    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013985    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013989    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013994    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.013999    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014004    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014009    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014013    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014020    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014026    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014030    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014035    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014615    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014624    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014629    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014634    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:04:18.019028 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014639    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014643    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014648    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014652    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014657    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014661    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014665    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014670    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014674    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014679    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014684    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014689    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014694    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014698    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014703    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014707    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014711    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014716    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014723    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:04:18.019588 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014728    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014757    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014763    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014774    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014778    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014782    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014786    2564 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014790    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014795    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014799    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014804    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014808    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014812    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014816    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014820    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014825    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014829    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014832    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014838    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014842    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:04:18.020081 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014847    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014851    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014856    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014861    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014865    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014869    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014874    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014878    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014882    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014888    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014892    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014897    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014901    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014907    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014912    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014916    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014921    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014925    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014929    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:04:18.020603 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014933    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014939    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014945    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014950    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014955    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014959    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014965    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014970    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014974    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014979    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014983    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014987    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014991    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.014995    2564 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015000    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015004    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015008    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015013    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015017    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:04:18.021089 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015022    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015027    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015031    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015035    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015039    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015044    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015048    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015053    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015057    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015062    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015066    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015073    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015077    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015081    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015085    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015089    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015093    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.015097    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015194    2564 flags.go:64] FLAG: --address="0.0.0.0"
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015204    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015214    2564 flags.go:64] FLAG: --anonymous-auth="true"
Aug 03 16:04:18.021563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015221    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015227    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015232    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015238    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015245    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015250    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015255    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015264    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015268    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015273    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015277    2564 flags.go:64] FLAG: --cgroup-root=""
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015281    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015286    2564 flags.go:64] FLAG: --client-ca-file=""
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015291    2564 flags.go:64] FLAG: --cloud-provider="external"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015295    2564 flags.go:64] FLAG: --cluster-dns="[]"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015304    2564 flags.go:64] FLAG: --cluster-domain=""
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015308    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015313    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015318    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015322    2564 flags.go:64] FLAG: --container-log-max-files="5"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015328    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015333    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015339    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 03 16:04:18.022160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015345    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015349    2564 flags.go:64] FLAG: --contention-profiling="false"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015354    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015359    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015363    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015368    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015374    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015380    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015384    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015389    2564 flags.go:64] FLAG: --enable-load-reader="false"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015394    2564 flags.go:64] FLAG: --enable-server="true"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015398    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015405    2564 flags.go:64] FLAG: --event-burst="100"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015409    2564 flags.go:64] FLAG: --event-qps="50"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015414    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015419    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015425    2564 flags.go:64] FLAG: --eviction-hard=""
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015432    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015437    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015442    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015447    2564 flags.go:64] FLAG: --eviction-soft=""
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015451    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015456    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015462    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015467    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 03 16:04:18.022711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015472    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015476    2564 flags.go:64] FLAG: --fail-swap-on="true"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015481    2564 flags.go:64] FLAG: --feature-gates=""
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015487    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015492    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015496    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015503    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015508    2564 flags.go:64] FLAG: --healthz-port="10248"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015513    2564 flags.go:64] FLAG: --help="false"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015518    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015523    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015528    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015533    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015538    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015545    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015549    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015554    2564 flags.go:64] FLAG: --image-service-endpoint=""
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015559    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015563    2564 flags.go:64] FLAG: --kube-api-burst="100"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015568    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015573    2564 flags.go:64] FLAG: --kube-api-qps="50"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015578    2564 flags.go:64] FLAG: --kube-reserved=""
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015583    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015590    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 03 16:04:18.023345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015595    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015600    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015605    2564 flags.go:64] FLAG: --lock-file=""
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015610    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015614    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015619    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015628    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015634    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015639    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015643    2564 flags.go:64] FLAG: --logging-format="text"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015648    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015653    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015657    2564 flags.go:64] FLAG: --manifest-url=""
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015662    2564 flags.go:64] FLAG: --manifest-url-header=""
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015670    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015675    2564 flags.go:64] FLAG: --max-open-files="1000000"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015682    2564 flags.go:64] FLAG: --max-pods="110"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015687    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015691    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015696    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015701    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015706    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015711    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015715    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015727    2564 flags.go:64] FLAG: --node-status-max-images="50"
Aug 03 16:04:18.023959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015750    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015756    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015761    2564 flags.go:64] FLAG: --pod-cidr=""
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015766    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015775    2564 flags.go:64] FLAG: --pod-manifest-path=""
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015780    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015787    2564 flags.go:64] FLAG: --pods-per-core="0"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015791    2564 flags.go:64] FLAG: --port="10250"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015796    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015801    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ca9d31763c5ac4ea"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015806    2564 flags.go:64] FLAG: --qos-reserved=""
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015812    2564 flags.go:64] FLAG: --read-only-port="10255"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015817    2564 flags.go:64] FLAG: --register-node="true"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015822    2564 flags.go:64] FLAG: --register-with-taints=""
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015827    2564 flags.go:64] FLAG: --registry-burst="10"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015833    2564 flags.go:64] FLAG: --registry-qps="5"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015838    2564 flags.go:64] FLAG: --reserved-cpus=""
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015843    2564 flags.go:64] FLAG: --reserved-memory=""
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015849    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015854    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015859    2564 flags.go:64] FLAG: --rotate-certificates="false"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015865    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015870    2564 flags.go:64] FLAG: --runonce="false"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015875    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015880    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 03 16:04:18.024598 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015885    2564 flags.go:64] FLAG: --seccomp-default="false"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015890    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015894    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015899    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015904    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015908    2564 flags.go:64] FLAG: --storage-driver-password="root"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015913    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015918    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015923    2564 flags.go:64] FLAG: --storage-driver-user="root"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015927    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015932    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015937    2564 flags.go:64] FLAG: --system-cgroups=""
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015942    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015953    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015957    2564 flags.go:64] FLAG: --tls-cert-file=""
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015962    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015970    2564 flags.go:64] FLAG: --tls-min-version=""
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015975    2564 flags.go:64] FLAG: --tls-private-key-file=""
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015980    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015985    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015989    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.015994    2564 flags.go:64] FLAG: --v="2"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.016001    2564 flags.go:64] FLAG: --version="false"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.016009    2564 flags.go:64] FLAG: --vmodule=""
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.016014    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 03 16:04:18.025228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.016020    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016223    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016230    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016235    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016240    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016245    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016249    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016254    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016258    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016263    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016267    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016271    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016276    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016280    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016285    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016289    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016293    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016298    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016302    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016306    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:04:18.025853 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016312    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016316    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016320    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016324    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016329    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016333    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016337    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016342    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016348    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016354    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016359    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016368    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016373    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016378    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016383    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016388    2564 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016392    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016396    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016400    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:04:18.026337 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016404    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016409    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016413    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016417    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016422    2564 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016426    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016430    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016434    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016438    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016442    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016446    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016450    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016455    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016460    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016465    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016469    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016473    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016477    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016481    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016485    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:04:18.026847 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016489    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016493    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016497    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016501    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016505    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016510    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016514    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016518    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016525    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016531    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016535    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016540    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016545    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016549    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016554    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016558    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016562    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016566    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016570    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:04:18.027331 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016574    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016579    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016583    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016587    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016591    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016596    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016601    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016605    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016610    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016614    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016618    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016623    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016627    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016631    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016636    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016640    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016644    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016648    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016652    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016659    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:04:18.027812 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016665    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.016669    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.016683    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.023751    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.023766    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.024492    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.024499    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024563    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024569    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024572    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024576    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024578    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024581    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024584    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024586    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:04:18.028298 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024589    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024592    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024595    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024597    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024600    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024603    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024606    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024610    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024614    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024619    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024623    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024626    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024629    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024632    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024634    2564 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024637    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024640    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024643    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024646    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:04:18.028663 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024649    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024651    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024656    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024659    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024662    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024665    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024668    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024671    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024674    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024677    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024679    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024682    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024684    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024687    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024689    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024692    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024695    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024697    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024701    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:04:18.029188 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024705    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024708    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024711    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024714    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024717    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024719    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024722    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024725    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024728    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024731    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024748    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024751    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024754    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024757    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024760    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024769    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024778    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024781    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024783    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024786    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:04:18.029648 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024788    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024791    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024793    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024796    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024798    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024801    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024803    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024806    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024809    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024811    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024814    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024816    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024819    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024821    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024824    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024827    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024829    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024832    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024834    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024837    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:04:18.030191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024839    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024842    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024844    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024846    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024849    2564 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024852    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024855    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024858    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024860    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024868    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024871    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024873    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.024876    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.024881    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025025    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:04:18.030666 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025030    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025033    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025036    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025039    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025042    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025045    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025047    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025050    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025052    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025055    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025057    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025060    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025062    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025065    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025068    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025070    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025075    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025078    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025081    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:04:18.031061 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025084    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025087    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025089    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025092    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025095    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025098    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025100    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025103    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025111    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025115    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025119    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025122    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025124    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025127    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025131    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025133    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025136    2564 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025139    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025141    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025144    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:04:18.031524 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025146    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025149    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025151    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025154    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025157    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025159    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025163    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025167    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025169    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025172    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025175    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025177    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025180    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025182    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025185    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025187    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025190    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025193    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025195    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:04:18.032017 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025198    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025200    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025210    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025213    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025215    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025218    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025220    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025222    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025225    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025227    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025230    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025233    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025235    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025238    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025241    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025243    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025246    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025248    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025251    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:04:18.032471 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025254    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025256    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025259    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025262    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025264    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025267    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025269    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025272    2564 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025275    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025277    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025280    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025282    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025285    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025287    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025290    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025292    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025300    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025303    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025306    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025308    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:04:18.032979 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:18.025310    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:04:18.033498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.025315    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 16:04:18.033498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.025430    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 03 16:04:18.033498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.028865    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 03 16:04:18.033498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.030064    2564 server.go:1019] "Starting client certificate rotation"
Aug 03 16:04:18.033498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.030158    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 03 16:04:18.033498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.030946    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 03 16:04:18.033498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.032670    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 03 16:04:18.039238 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.039212    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 03 16:04:18.050460 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.050441    2564 log.go:25] "Validated CRI v1 runtime API"
Aug 03 16:04:18.052312 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.052294    2564 log.go:25] "Validated CRI v1 image API"
Aug 03 16:04:18.053158 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.053140    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 03 16:04:18.056721 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.056695    2564 fs.go:135] Filesystem UUIDs: map[1ddb03b8-6b36-4a9a-8785-7d86f187f21b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 975f8371-7c24-4396-8ae0-2093f17c8867:/dev/nvme0n1p3]
Aug 03 16:04:18.056804 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.056721    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 03 16:04:18.061777 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.061647    2564 manager.go:216] Machine: {Timestamp:2026-08-03 16:04:18.060450724 +0000 UTC m=+0.327266722 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098999 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26dd202d5f91f2719aff6d3bfb85da SystemUUID:ec26dd20-2d5f-91f2-719a-ff6d3bfb85da BootID:2ca4b652-301d-409b-b19c-a7ded502380b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e4:60:e0:1f:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e4:60:e0:1f:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:11:25:b4:b6:c6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 03 16:04:18.061777 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.061768    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 03 16:04:18.061896 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.061849    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 03 16:04:18.062876 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.062858    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 03 16:04:18.064886 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.064858    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 03 16:04:18.065062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.064894    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 03 16:04:18.066066 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.066053    2564 topology_manager.go:138] "Creating topology manager with none policy"
Aug 03 16:04:18.066124 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.066074    2564 container_manager_linux.go:309] "Creating device plugin manager"
Aug 03 16:04:18.066124 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.066085    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 03 16:04:18.067686 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.067673    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 03 16:04:18.068813 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.068800    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 03 16:04:18.072106 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072092    2564 state_mem.go:36] "Initialized new in-memory state store"
Aug 03 16:04:18.072237 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072226    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 03 16:04:18.072304 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072263    2564 kubelet.go:486] "Attempting to sync node with API server"
Aug 03 16:04:18.072304 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072277    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 03 16:04:18.072304 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072296    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 03 16:04:18.072422 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072309    2564 kubelet.go:398] "Adding apiserver pod source"
Aug 03 16:04:18.072422 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072321    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 03 16:04:18.072701 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072687    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 03 16:04:18.072775 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.072710    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 03 16:04:18.075450 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.075432    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 03 16:04:18.076714 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.076701    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 03 16:04:18.076792 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.076730    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 03 16:04:18.077305 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077294    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 03 16:04:18.077340 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077312    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 03 16:04:18.077340 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077319    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 03 16:04:18.077340 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077324    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077345    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077351    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077359    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077364    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077371    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077376    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077388    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 03 16:04:18.077420 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.077393    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 03 16:04:18.078047 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.078037    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 03 16:04:18.078047 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.078047    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 03 16:04:18.081063 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.081048    2564 server.go:1268] "Started kubelet"
Aug 03 16:04:18.081176 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.081128    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 03 16:04:18.081231 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.081198    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 03 16:04:18.081892 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.081591    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 03 16:04:18.082259 ip-10-0-138-110 systemd[1]: Started Kubernetes Kubelet.
Aug 03 16:04:18.082525 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.082504    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 03 16:04:18.084254 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.084238    2564 server.go:310] "Adding debug handlers to kubelet server"
Aug 03 16:04:18.085512 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.085492    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-110.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 03 16:04:18.085811 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.085784    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 03 16:04:18.086066 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.086046    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 03 16:04:18.088144 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.088121    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 03 16:04:18.088626 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.088609    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7ld7w"
Aug 03 16:04:18.089909 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.089895    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 03 16:04:18.089980 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.089907    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 03 16:04:18.090033 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.089963    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.090431 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.090412    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 03 16:04:18.090520 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.090448    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 03 16:04:18.090520 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.090464    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 03 16:04:18.090627 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.090577    2564 reconstruct.go:97] "Volume reconstruction finished"
Aug 03 16:04:18.090627 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.090583    2564 reconciler.go:29] "Reconciler: start to sync state"
Aug 03 16:04:18.091222 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.090335    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-110.ec2.internal.18c857228861fd4d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-110.ec2.internal,UID:ip-10-0-138-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-110.ec2.internal,},FirstTimestamp:2026-08-03 16:04:18.081029453 +0000 UTC m=+0.347845450,LastTimestamp:2026-08-03 16:04:18.081029453 +0000 UTC m=+0.347845450,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-110.ec2.internal,}"
Aug 03 16:04:18.091569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.091553    2564 factory.go:55] Registering systemd factory
Aug 03 16:04:18.091569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.091569    2564 factory.go:223] Registration of the systemd container factory successfully
Aug 03 16:04:18.091844 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.091828    2564 factory.go:153] Registering CRI-O factory
Aug 03 16:04:18.091844 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.091845    2564 factory.go:223] Registration of the crio container factory successfully
Aug 03 16:04:18.091939 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.091895    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 03 16:04:18.091939 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.091913    2564 factory.go:103] Registering Raw factory
Aug 03 16:04:18.091939 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.091923    2564 manager.go:1196] Started watching for new ooms in manager
Aug 03 16:04:18.092310 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.092297    2564 manager.go:349] Starting recovery of all containers
Aug 03 16:04:18.097370 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.097347    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7ld7w"
Aug 03 16:04:18.102002 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.101974    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 03 16:04:18.102093 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.102004    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 03 16:04:18.103909 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.103895    2564 manager.go:354] Recovery completed
Aug 03 16:04:18.109565 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.109553    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:04:18.111413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.111399    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:04:18.111472 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.111425    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:04:18.111472 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.111444    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:04:18.112022 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.112005    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 03 16:04:18.112104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.112023    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 03 16:04:18.112104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.112025    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:04:18.112104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.112042    2564 state_mem.go:36] "Initialized new in-memory state store"
Aug 03 16:04:18.112745 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.112721    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:04:18.112834 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.112776    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:04:18.112834 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.112794    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:04:18.113531 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.113519    2564 policy_none.go:49] "None policy: Start"
Aug 03 16:04:18.113600 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.113537    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 03 16:04:18.113600 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.113550    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 03 16:04:18.114775 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.114755    2564 policy_none.go:47] "Start"
Aug 03 16:04:18.153431 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.153255    2564 manager.go:337] "Starting Device Plugin manager"
Aug 03 16:04:18.153547 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.153458    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 03 16:04:18.153547 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.153472    2564 server.go:87] "Starting device plugin registration server"
Aug 03 16:04:18.153747 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.153723    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 03 16:04:18.153793 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.153759    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 03 16:04:18.153885 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.153869    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 03 16:04:18.153979 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.153943    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 03 16:04:18.153979 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.153956    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 03 16:04:18.154510 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.154492    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 03 16:04:18.154578 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.154546    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.236146 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.236111    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 03 16:04:18.237250 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.237234    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 03 16:04:18.237338 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.237271    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 03 16:04:18.237338 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.237293    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 03 16:04:18.237338 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.237331    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 03 16:04:18.239267 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.239244    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:18.254768 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.254748    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:04:18.255531 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.255515    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:04:18.255604 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.255544    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:04:18.255604 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.255557    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:04:18.255604 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.255577    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.262753 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.262706    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.262753 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.262724    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-110.ec2.internal\": node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.277932 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.277916    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.337594 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.337557    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal"]
Aug 03 16:04:18.337694 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.337649    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:04:18.339175 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.339160    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:04:18.339240 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.339188    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:04:18.339240 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.339198    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:04:18.341202 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.341190    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:04:18.341340 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.341328    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.341377 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.341358    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:04:18.341937 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.341922    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:04:18.342001 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.341947    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:04:18.342001 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.341960    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:04:18.342062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.341961    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:04:18.342062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.342028    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:04:18.342062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.342039    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:04:18.343887 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.343872    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.343946 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.343893    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:04:18.344497 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.344482    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:04:18.344554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.344508    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:04:18.344554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.344517    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:04:18.370304 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.370285    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-110.ec2.internal\" not found" node="ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.374451 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.374436    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-110.ec2.internal\" not found" node="ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.378693 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.378679    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.479568 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.479535    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.491796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.491777    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d96742d16eb1498950aa8db69ed93ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal\" (UID: \"9d96742d16eb1498950aa8db69ed93ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.491866 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.491800    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d96742d16eb1498950aa8db69ed93ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal\" (UID: \"9d96742d16eb1498950aa8db69ed93ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.491866 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.491816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab1f6e49c95683c2cc3b4fd509ec7a98-config\") pod \"kube-apiserver-proxy-ip-10-0-138-110.ec2.internal\" (UID: \"ab1f6e49c95683c2cc3b4fd509ec7a98\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.580056 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.580029    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.593286 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.592353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d96742d16eb1498950aa8db69ed93ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal\" (UID: \"9d96742d16eb1498950aa8db69ed93ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.593286 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.592382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab1f6e49c95683c2cc3b4fd509ec7a98-config\") pod \"kube-apiserver-proxy-ip-10-0-138-110.ec2.internal\" (UID: \"ab1f6e49c95683c2cc3b4fd509ec7a98\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.593286 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.592403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d96742d16eb1498950aa8db69ed93ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal\" (UID: \"9d96742d16eb1498950aa8db69ed93ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.593286 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.592451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d96742d16eb1498950aa8db69ed93ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal\" (UID: \"9d96742d16eb1498950aa8db69ed93ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.593286 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.592457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab1f6e49c95683c2cc3b4fd509ec7a98-config\") pod \"kube-apiserver-proxy-ip-10-0-138-110.ec2.internal\" (UID: \"ab1f6e49c95683c2cc3b4fd509ec7a98\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.593286 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.592459    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d96742d16eb1498950aa8db69ed93ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal\" (UID: \"9d96742d16eb1498950aa8db69ed93ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.672481 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.672447    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.677609 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:18.677155    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:18.680211 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.680190    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.780436 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.780406    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.880862 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.880791    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:18.981109 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:18.981088    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:19.029506 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.029482    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 03 16:04:19.029908 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.029608    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 03 16:04:19.081904 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:19.081866    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:19.101847 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.101812    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-02 15:59:17 +0000 UTC" deadline="2028-01-09 23:30:30.356729012 +0000 UTC"
Aug 03 16:04:19.101847 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.101842    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12583h26m11.254889446s"
Aug 03 16:04:19.182247 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:19.182167    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:19.279882 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.279857    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:19.282796 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:19.282780    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:19.383223 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:19.383190    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:19.483605 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:19.483549    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:19.519074 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.519047    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:19.584600 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:19.584564    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-110.ec2.internal\" not found"
Aug 03 16:04:19.632597 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.632572    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:19.690719 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.690700    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:19.698125 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.698109    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 03 16:04:19.699235 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.699224    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal"
Aug 03 16:04:19.707930 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:19.707915    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 03 16:04:20.073376 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.073354    2564 apiserver.go:52] "Watching apiserver"
Aug 03 16:04:20.081865 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.081836    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 03 16:04:20.082165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.082146    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-n989x","openshift-image-registry/node-ca-69qfd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal","openshift-multus/multus-g9mzt","openshift-multus/network-metrics-daemon-sb5nd","kube-system/global-pull-secret-syncer-5nw44","kube-system/konnectivity-agent-jkhtn","kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal","openshift-dns/node-resolver-2dtgs","openshift-multus/multus-additional-cni-plugins-z5qrk","openshift-network-diagnostics/network-check-target-drsdv","openshift-network-operator/iptables-alerter-lrmsx","openshift-ovn-kubernetes/ovnkube-node-mprtd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"]
Aug 03 16:04:20.084328 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.084310    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.086077 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.086063    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.086458 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.086442    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kzcvl\""
Aug 03 16:04:20.086562 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.086546    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:20.086647 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.086632    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:20.087958 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.087943    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:20.088037 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.087991    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:20.088192 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.088178    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 03 16:04:20.088329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.088315    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8skqn\""
Aug 03 16:04:20.088377 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.088340    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 03 16:04:20.088616 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.088603    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 03 16:04:20.089977 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.089961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.091309 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.091294    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 03 16:04:20.091940 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.091927    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.092282 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.092266    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 03 16:04:20.092282 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.092277    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 03 16:04:20.092417 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.092299    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 03 16:04:20.092417 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.092271    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 03 16:04:20.092417 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.092362    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v8l4s\""
Aug 03 16:04:20.093893 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.093879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.094226 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.094213    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 03 16:04:20.094297 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.094255    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 03 16:04:20.094297 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.094213    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nll8c\""
Aug 03 16:04:20.095810 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.095796    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.095886 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.095868    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:20.096127 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.096114    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gm6s2\""
Aug 03 16:04:20.096379 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.096364    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 03 16:04:20.096445 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.096394    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 03 16:04:20.098147 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.098130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:20.098220 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.098197    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:20.100178 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e16925c5-9e44-4b01-bb5d-fd1172662359-tmp-dir\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.100259 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100185    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysctl-d\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.100259 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-serviceca\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.100259 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-cni-bin\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.100259 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-hostroot\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.100406 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-var-lib-kubelet\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.100702 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100277    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-host\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100784    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-cni-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100814    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-cni-multus\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100834    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysconfig\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-kubernetes\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100899    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-systemd\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.100943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-host\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101093    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101112    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-system-cni-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-os-release\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-k8s-cni-cncf-io\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-multus-certs\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhl8s\" (UniqueName: \"kubernetes.io/projected/e16925c5-9e44-4b01-bb5d-fd1172662359-kube-api-access-rhl8s\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysctl-conf\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-cnibin\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-sys\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09bca23f-ea74-4960-909e-4662dea6789b-tmp\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61322de4-f04c-4b27-902b-7bbddc29b6b7-cni-binary-copy\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-conf-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101469    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-modprobe-d\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101574    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-socket-dir-parent\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-netns\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-kubelet\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.101796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e16925c5-9e44-4b01-bb5d-fd1172662359-hosts-file\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.102268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-lib-modules\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.102268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09bca23f-ea74-4960-909e-4662dea6789b-etc-tuned\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.102268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.101991    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sshd5\" (UniqueName: \"kubernetes.io/projected/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-kube-api-access-sshd5\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.102268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.102186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr7nc\" (UniqueName: \"kubernetes.io/projected/61322de4-f04c-4b27-902b-7bbddc29b6b7-kube-api-access-wr7nc\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.102503 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.102487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-run\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.102560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.102536    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qnlh\" (UniqueName: \"kubernetes.io/projected/09bca23f-ea74-4960-909e-4662dea6789b-kube-api-access-9qnlh\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.102606 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.102571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-daemon-config\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.102640 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.102613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-etc-kubernetes\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.103772 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.103748    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 03 16:04:20.104179 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.104159    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.104274 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.104227    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kts2d\""
Aug 03 16:04:20.104337 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.104317    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 03 16:04:20.105274 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.105260    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 03 16:04:20.106359 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.106346    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.106359 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.106353    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x4ksf\""
Aug 03 16:04:20.106438 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.106357    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:20.106797 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.106784    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:20.106905 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.106892    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 03 16:04:20.108407 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.108393    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.108496 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.108482    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 03 16:04:20.108569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.108555    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 03 16:04:20.108799 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.108783    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 03 16:04:20.109954 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.109938    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 03 16:04:20.110112 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.109942    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 03 16:04:20.110112 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.110015    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ssntm\""
Aug 03 16:04:20.110112 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.109944    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 03 16:04:20.110500 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.110486    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 03 16:04:20.110574 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.110517    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 03 16:04:20.110627 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.110537    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-glj2d\""
Aug 03 16:04:20.110679 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.110641    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 03 16:04:20.122999 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.122986    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lfmtn"
Aug 03 16:04:20.130373 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.130357    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lfmtn"
Aug 03 16:04:20.191218 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.191201    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 03 16:04:20.203191 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203174    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09bca23f-ea74-4960-909e-4662dea6789b-etc-tuned\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.203255 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr7nc\" (UniqueName: \"kubernetes.io/projected/61322de4-f04c-4b27-902b-7bbddc29b6b7-kube-api-access-wr7nc\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.203255 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5drr\" (UniqueName: \"kubernetes.io/projected/1fe44827-6e6a-4c1e-a105-a6734ddbb548-kube-api-access-q5drr\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.203255 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts699\" (UniqueName: \"kubernetes.io/projected/b1474a27-5420-4708-9008-10067c3fddca-kube-api-access-ts699\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.203388 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-iptables-alerter-script\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.203388 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-etc-selinux\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.203388 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qnlh\" (UniqueName: \"kubernetes.io/projected/09bca23f-ea74-4960-909e-4662dea6789b-kube-api-access-9qnlh\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.203388 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203372    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-daemon-config\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.203388 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-etc-kubernetes\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.203584 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-dbus\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.203584 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203457    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 03 16:04:20.203584 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-etc-kubernetes\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.204228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-run-netns\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.204228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-serviceca\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.204228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.203996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovnkube-script-lib\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.204228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204031    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-host-slash\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.204228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-cni-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.204228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204084    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-cni-multus\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.204228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204210    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-kubernetes\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-host\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-kubernetes\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204291    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-daemon-config\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-multus-certs\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-systemd\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-host\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovn-node-metrics-cert\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-multus-certs\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204473    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-cni-multus\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.204560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204525    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-sys-fs\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204578    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-serviceca\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204632    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-etc-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-run-ovn-kubernetes\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g2jj\" (UniqueName: \"kubernetes.io/projected/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-kube-api-access-2g2jj\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-socket-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204777    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-cni-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-slash\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-socket-dir-parent\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.204916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-kubelet-config\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205055    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-socket-dir-parent\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.205144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205143    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovnkube-config\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-os-release\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205260    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-registration-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpvhj\" (UniqueName: \"kubernetes.io/projected/42f98018-4b08-482d-b36d-5caaa3b6b522-kube-api-access-lpvhj\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e16925c5-9e44-4b01-bb5d-fd1172662359-hosts-file\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sshd5\" (UniqueName: \"kubernetes.io/projected/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-kube-api-access-sshd5\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-run\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79aebf44-1875-4f4e-a708-c21df0257b1d-konnectivity-ca\") pod \"konnectivity-agent-jkhtn\" (UID: \"79aebf44-1875-4f4e-a708-c21df0257b1d\") " pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-system-cni-dir\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e16925c5-9e44-4b01-bb5d-fd1172662359-tmp-dir\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysctl-d\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-cni-bin\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205580    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-hostroot\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz5td\" (UniqueName: \"kubernetes.io/projected/9ab2f368-65f9-45e5-912b-c5d207c9ea22-kube-api-access-fz5td\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205641    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-node-log\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.207424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e16925c5-9e44-4b01-bb5d-fd1172662359-hosts-file\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-var-lib-kubelet\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.205982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e16925c5-9e44-4b01-bb5d-fd1172662359-tmp-dir\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206118    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysctl-d\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206184    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-cni-bin\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-hostroot\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-run\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-host\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-var-lib-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206442    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-cni-bin\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206468    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-cnibin\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206513    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-device-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysconfig\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206583    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-systemd\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysconfig\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206632    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-system-cni-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206639    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-var-lib-kubelet\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206688    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-os-release\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206703    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-system-cni-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-k8s-cni-cncf-io\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206754    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-systemd\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206799    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-os-release\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.206829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhl8s\" (UniqueName: \"kubernetes.io/projected/e16925c5-9e44-4b01-bb5d-fd1172662359-kube-api-access-rhl8s\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysctl-conf\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207144    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-cnibin\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09bca23f-ea74-4960-909e-4662dea6789b-etc-tuned\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207176    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-kubelet\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-log-socket\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207269    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79aebf44-1875-4f4e-a708-c21df0257b1d-agent-certs\") pod \"konnectivity-agent-jkhtn\" (UID: \"79aebf44-1875-4f4e-a708-c21df0257b1d\") " pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207288    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-sysctl-conf\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207301    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-sys\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-k8s-cni-cncf-io\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-cnibin\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.208854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09bca23f-ea74-4960-909e-4662dea6789b-tmp\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-sys\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61322de4-f04c-4b27-902b-7bbddc29b6b7-cni-binary-copy\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207444    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-conf-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207475    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-ovn\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207532    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-host\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-systemd-units\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-multus-conf-dir\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-modprobe-d\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-netns\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-kubelet\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-etc-modprobe-d\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-cni-netd\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207928    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61322de4-f04c-4b27-902b-7bbddc29b6b7-cni-binary-copy\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207921    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-run-netns\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.207925    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-env-overrides\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.208002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61322de4-f04c-4b27-902b-7bbddc29b6b7-host-var-lib-kubelet\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.209529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.208071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-lib-modules\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.210135 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.208249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09bca23f-ea74-4960-909e-4662dea6789b-lib-modules\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.210135 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.209399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09bca23f-ea74-4960-909e-4662dea6789b-tmp\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.211648 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.211633    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:20.211648 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.211649    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:20.211804 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.211666    2564 projected.go:196] Error preparing data for projected volume kube-api-access-l2vhk for pod openshift-network-diagnostics/network-check-target-drsdv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:20.211804 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.211724    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk podName:d377dce8-488f-49f1-9e16-9f661e0aba4a nodeName:}" failed. No retries permitted until 2026-08-03 16:04:20.711711682 +0000 UTC m=+2.978527683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l2vhk" (UniqueName: "kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk") pod "network-check-target-drsdv" (UID: "d377dce8-488f-49f1-9e16-9f661e0aba4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:20.213666 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.213631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr7nc\" (UniqueName: \"kubernetes.io/projected/61322de4-f04c-4b27-902b-7bbddc29b6b7-kube-api-access-wr7nc\") pod \"multus-g9mzt\" (UID: \"61322de4-f04c-4b27-902b-7bbddc29b6b7\") " pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.213774 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.213686    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sshd5\" (UniqueName: \"kubernetes.io/projected/c244ee48-8f52-4a87-92cd-df1c00b2d8ca-kube-api-access-sshd5\") pod \"node-ca-69qfd\" (UID: \"c244ee48-8f52-4a87-92cd-df1c00b2d8ca\") " pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.213774 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.213690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qnlh\" (UniqueName: \"kubernetes.io/projected/09bca23f-ea74-4960-909e-4662dea6789b-kube-api-access-9qnlh\") pod \"tuned-n989x\" (UID: \"09bca23f-ea74-4960-909e-4662dea6789b\") " pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.213976 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.213959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhl8s\" (UniqueName: \"kubernetes.io/projected/e16925c5-9e44-4b01-bb5d-fd1172662359-kube-api-access-rhl8s\") pod \"node-resolver-2dtgs\" (UID: \"e16925c5-9e44-4b01-bb5d-fd1172662359\") " pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.274087 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.274065    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:20.309155 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79aebf44-1875-4f4e-a708-c21df0257b1d-konnectivity-ca\") pod \"konnectivity-agent-jkhtn\" (UID: \"79aebf44-1875-4f4e-a708-c21df0257b1d\") " pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-system-cni-dir\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309207    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz5td\" (UniqueName: \"kubernetes.io/projected/9ab2f368-65f9-45e5-912b-c5d207c9ea22-kube-api-access-fz5td\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-node-log\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-node-log\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309300    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-system-cni-dir\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-var-lib-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309373    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-cni-bin\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-var-lib-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-cnibin\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-cni-bin\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-device-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-kubelet\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-cnibin\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-log-socket\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-device-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309563    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-kubelet\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79aebf44-1875-4f4e-a708-c21df0257b1d-agent-certs\") pod \"konnectivity-agent-jkhtn\" (UID: \"79aebf44-1875-4f4e-a708-c21df0257b1d\") " pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309607    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-ovn\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-log-socket\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.309716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309636    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-systemd-units\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-ovn\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-systemd-units\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-cni-netd\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309710    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79aebf44-1875-4f4e-a708-c21df0257b1d-konnectivity-ca\") pod \"konnectivity-agent-jkhtn\" (UID: \"79aebf44-1875-4f4e-a708-c21df0257b1d\") " pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.309769    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309770    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-cni-netd\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309772    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-env-overrides\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5drr\" (UniqueName: \"kubernetes.io/projected/1fe44827-6e6a-4c1e-a105-a6734ddbb548-kube-api-access-q5drr\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts699\" (UniqueName: \"kubernetes.io/projected/b1474a27-5420-4708-9008-10067c3fddca-kube-api-access-ts699\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.309845    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs podName:9ab2f368-65f9-45e5-912b-c5d207c9ea22 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:20.809826803 +0000 UTC m=+3.076642810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs") pod "network-metrics-daemon-sb5nd" (UID: "9ab2f368-65f9-45e5-912b-c5d207c9ea22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-iptables-alerter-script\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-etc-selinux\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-dbus\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.310513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.309969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-run-netns\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovnkube-script-lib\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-host-slash\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310031    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-etc-selinux\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310043    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-dbus\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310092    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-run-netns\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-systemd\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovn-node-metrics-cert\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310135    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-run-systemd\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-host-slash\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310152    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-env-overrides\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-sys-fs\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-etc-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-etc-openvswitch\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-sys-fs\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.311368 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.310336    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.310378    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret podName:c18f9f7f-3374-4214-b9e8-a98a23f5cc98 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:20.810362842 +0000 UTC m=+3.077178839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret") pod "global-pull-secret-syncer-5nw44" (UID: "c18f9f7f-3374-4214-b9e8-a98a23f5cc98") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310400    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-run-ovn-kubernetes\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-iptables-alerter-script\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g2jj\" (UniqueName: \"kubernetes.io/projected/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-kube-api-access-2g2jj\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-socket-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-slash\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-kubelet-config\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-socket-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovnkube-config\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovnkube-script-lib\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-run-ovn-kubernetes\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310609    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1fe44827-6e6a-4c1e-a105-a6734ddbb548-host-slash\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310600    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-os-release\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.312105 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310647    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-kubelet-config\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-registration-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpvhj\" (UniqueName: \"kubernetes.io/projected/42f98018-4b08-482d-b36d-5caaa3b6b522-kube-api-access-lpvhj\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310750    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1474a27-5420-4708-9008-10067c3fddca-os-release\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.310775    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42f98018-4b08-482d-b36d-5caaa3b6b522-registration-dir\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.311061    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovnkube-config\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.311452    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1474a27-5420-4708-9008-10067c3fddca-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.312219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79aebf44-1875-4f4e-a708-c21df0257b1d-agent-certs\") pod \"konnectivity-agent-jkhtn\" (UID: \"79aebf44-1875-4f4e-a708-c21df0257b1d\") " pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.312516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.312343    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1fe44827-6e6a-4c1e-a105-a6734ddbb548-ovn-node-metrics-cert\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.317919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.317895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g2jj\" (UniqueName: \"kubernetes.io/projected/623f60cf-b75a-47d9-bd9d-5d4f954ac74e-kube-api-access-2g2jj\") pod \"iptables-alerter-lrmsx\" (UID: \"623f60cf-b75a-47d9-bd9d-5d4f954ac74e\") " pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.318301 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.318281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5drr\" (UniqueName: \"kubernetes.io/projected/1fe44827-6e6a-4c1e-a105-a6734ddbb548-kube-api-access-q5drr\") pod \"ovnkube-node-mprtd\" (UID: \"1fe44827-6e6a-4c1e-a105-a6734ddbb548\") " pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.318383 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.318369    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz5td\" (UniqueName: \"kubernetes.io/projected/9ab2f368-65f9-45e5-912b-c5d207c9ea22-kube-api-access-fz5td\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:20.318536 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.318522    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts699\" (UniqueName: \"kubernetes.io/projected/b1474a27-5420-4708-9008-10067c3fddca-kube-api-access-ts699\") pod \"multus-additional-cni-plugins-z5qrk\" (UID: \"b1474a27-5420-4708-9008-10067c3fddca\") " pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.318846 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.318833    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpvhj\" (UniqueName: \"kubernetes.io/projected/42f98018-4b08-482d-b36d-5caaa3b6b522-kube-api-access-lpvhj\") pod \"aws-ebs-csi-driver-node-jd5t7\" (UID: \"42f98018-4b08-482d-b36d-5caaa3b6b522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.413903 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.413841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n989x"
Aug 03 16:04:20.419245 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.419222    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-69qfd"
Aug 03 16:04:20.423908 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.423889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9mzt"
Aug 03 16:04:20.434978 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.434944    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bca23f_ea74_4960_909e_4662dea6789b.slice/crio-4809170fa5d2a66488c5bf71f9c5c8933ee26897384e848dcd1cdec1049243c1 WatchSource:0}: Error finding container 4809170fa5d2a66488c5bf71f9c5c8933ee26897384e848dcd1cdec1049243c1: Status 404 returned error can't find the container with id 4809170fa5d2a66488c5bf71f9c5c8933ee26897384e848dcd1cdec1049243c1
Aug 03 16:04:20.435397 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.435378    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61322de4_f04c_4b27_902b_7bbddc29b6b7.slice/crio-f0e968177030592c927f2ca389aff56ad7399aa06cd424a04b3e692b485fde52 WatchSource:0}: Error finding container f0e968177030592c927f2ca389aff56ad7399aa06cd424a04b3e692b485fde52: Status 404 returned error can't find the container with id f0e968177030592c927f2ca389aff56ad7399aa06cd424a04b3e692b485fde52
Aug 03 16:04:20.436133 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.436116    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d96742d16eb1498950aa8db69ed93ac.slice/crio-c0a353ee9edff1ab4c58b84d2b4b31de32c494a51dbc62abd12e5105e9b978cd WatchSource:0}: Error finding container c0a353ee9edff1ab4c58b84d2b4b31de32c494a51dbc62abd12e5105e9b978cd: Status 404 returned error can't find the container with id c0a353ee9edff1ab4c58b84d2b4b31de32c494a51dbc62abd12e5105e9b978cd
Aug 03 16:04:20.436725 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.436708    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1f6e49c95683c2cc3b4fd509ec7a98.slice/crio-e612723a6b2333a9618c6d6dc736f7aef57b1dea88f29d24755dc03ebe50c0a8 WatchSource:0}: Error finding container e612723a6b2333a9618c6d6dc736f7aef57b1dea88f29d24755dc03ebe50c0a8: Status 404 returned error can't find the container with id e612723a6b2333a9618c6d6dc736f7aef57b1dea88f29d24755dc03ebe50c0a8
Aug 03 16:04:20.437272 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.437232    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc244ee48_8f52_4a87_92cd_df1c00b2d8ca.slice/crio-422560cff467fb1cea18476aab3b8b62c77890c2a90aee9031c7e5fcade757e2 WatchSource:0}: Error finding container 422560cff467fb1cea18476aab3b8b62c77890c2a90aee9031c7e5fcade757e2: Status 404 returned error can't find the container with id 422560cff467fb1cea18476aab3b8b62c77890c2a90aee9031c7e5fcade757e2
Aug 03 16:04:20.441808 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.440677    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:04:20.444398 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.444382    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2dtgs"
Aug 03 16:04:20.452930 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.452910    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16925c5_9e44_4b01_bb5d_fd1172662359.slice/crio-986b451067ac541a7796e0dc10dfa0c297e7e5f60d2e0956ad64b6656221236d WatchSource:0}: Error finding container 986b451067ac541a7796e0dc10dfa0c297e7e5f60d2e0956ad64b6656221236d: Status 404 returned error can't find the container with id 986b451067ac541a7796e0dc10dfa0c297e7e5f60d2e0956ad64b6656221236d
Aug 03 16:04:20.474311 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.474286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:20.482168 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.482144    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79aebf44_1875_4f4e_a708_c21df0257b1d.slice/crio-26a4858abfb6d69680ed3f507375d0312bc067322ce889ae4a7f00bbb149b6fc WatchSource:0}: Error finding container 26a4858abfb6d69680ed3f507375d0312bc067322ce889ae4a7f00bbb149b6fc: Status 404 returned error can't find the container with id 26a4858abfb6d69680ed3f507375d0312bc067322ce889ae4a7f00bbb149b6fc
Aug 03 16:04:20.490600 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.490580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z5qrk"
Aug 03 16:04:20.497612 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.497589    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1474a27_5420_4708_9008_10067c3fddca.slice/crio-8306fae755aaf7d51b99467760b0e93bccb315cde6abab943181d8edfc5c6d64 WatchSource:0}: Error finding container 8306fae755aaf7d51b99467760b0e93bccb315cde6abab943181d8edfc5c6d64: Status 404 returned error can't find the container with id 8306fae755aaf7d51b99467760b0e93bccb315cde6abab943181d8edfc5c6d64
Aug 03 16:04:20.514818 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.514798    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lrmsx"
Aug 03 16:04:20.522684 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.522664    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623f60cf_b75a_47d9_bd9d_5d4f954ac74e.slice/crio-960330361c2ebe2492bd38009584f8368c89f904db19f8972cd1e42fd56ec43b WatchSource:0}: Error finding container 960330361c2ebe2492bd38009584f8368c89f904db19f8972cd1e42fd56ec43b: Status 404 returned error can't find the container with id 960330361c2ebe2492bd38009584f8368c89f904db19f8972cd1e42fd56ec43b
Aug 03 16:04:20.536265 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.536242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:20.540939 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.540925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7"
Aug 03 16:04:20.543256 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.543233    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe44827_6e6a_4c1e_a105_a6734ddbb548.slice/crio-032e96e7394a82bb3f05dccc0430d37f8581774d52883d71838e20f24397d7b2 WatchSource:0}: Error finding container 032e96e7394a82bb3f05dccc0430d37f8581774d52883d71838e20f24397d7b2: Status 404 returned error can't find the container with id 032e96e7394a82bb3f05dccc0430d37f8581774d52883d71838e20f24397d7b2
Aug 03 16:04:20.548229 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:04:20.548211    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f98018_4b08_482d_b36d_5caaa3b6b522.slice/crio-47e56042c6fab55a479e1fff0f4f2d741ca735374418fcf845853912e5f5f9a5 WatchSource:0}: Error finding container 47e56042c6fab55a479e1fff0f4f2d741ca735374418fcf845853912e5f5f9a5: Status 404 returned error can't find the container with id 47e56042c6fab55a479e1fff0f4f2d741ca735374418fcf845853912e5f5f9a5
Aug 03 16:04:20.549552 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.549536    2564 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-jd5t7_openshift-cluster-csi-drivers(42f98018-4b08-482d-b36d-5caaa3b6b522): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Aug 03 16:04:20.550180 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.550165    2564 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-jd5t7_openshift-cluster-csi-drivers(42f98018-4b08-482d-b36d-5caaa3b6b522): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Aug 03 16:04:20.551838 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.551822    2564 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-jd5t7_openshift-cluster-csi-drivers(42f98018-4b08-482d-b36d-5caaa3b6b522): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Aug 03 16:04:20.551892 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.551858    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7" podUID="42f98018-4b08-482d-b36d-5caaa3b6b522"
Aug 03 16:04:20.714721 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.714650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:20.714868 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.714809    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:20.714868 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.714825    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:20.714868 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.714834    2564 projected.go:196] Error preparing data for projected volume kube-api-access-l2vhk for pod openshift-network-diagnostics/network-check-target-drsdv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:20.714975 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.714880    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk podName:d377dce8-488f-49f1-9e16-9f661e0aba4a nodeName:}" failed. No retries permitted until 2026-08-03 16:04:21.714864556 +0000 UTC m=+3.981680552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l2vhk" (UniqueName: "kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk") pod "network-check-target-drsdv" (UID: "d377dce8-488f-49f1-9e16-9f661e0aba4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:20.815731 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.815700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:20.815925 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:20.815758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:20.815925 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.815844    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:20.815925 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.815847    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:20.815925 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.815895    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret podName:c18f9f7f-3374-4214-b9e8-a98a23f5cc98 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:21.815882785 +0000 UTC m=+4.082698767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret") pod "global-pull-secret-syncer-5nw44" (UID: "c18f9f7f-3374-4214-b9e8-a98a23f5cc98") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:20.815925 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:20.815907    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs podName:9ab2f368-65f9-45e5-912b-c5d207c9ea22 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:21.81590141 +0000 UTC m=+4.082717393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs") pod "network-metrics-daemon-sb5nd" (UID: "9ab2f368-65f9-45e5-912b-c5d207c9ea22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:21.133792 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.133674    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 15:59:19 +0000 UTC" deadline="2028-01-21 13:38:22.096338867 +0000 UTC"
Aug 03 16:04:21.133792 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.133729    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12861h34m0.962619413s"
Aug 03 16:04:21.238091 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.237615    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:21.238091 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.237744    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:21.251983 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.251923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal" event={"ID":"ab1f6e49c95683c2cc3b4fd509ec7a98","Type":"ContainerStarted","Data":"e612723a6b2333a9618c6d6dc736f7aef57b1dea88f29d24755dc03ebe50c0a8"}
Aug 03 16:04:21.263762 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.263717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7" event={"ID":"42f98018-4b08-482d-b36d-5caaa3b6b522","Type":"ContainerStarted","Data":"47e56042c6fab55a479e1fff0f4f2d741ca735374418fcf845853912e5f5f9a5"}
Aug 03 16:04:21.275232 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.275196    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2dtgs" event={"ID":"e16925c5-9e44-4b01-bb5d-fd1172662359","Type":"ContainerStarted","Data":"986b451067ac541a7796e0dc10dfa0c297e7e5f60d2e0956ad64b6656221236d"}
Aug 03 16:04:21.277178 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.277139    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98f7a67308f5a1349c0b76014caaadd1bd695bcc6fa6b230ca629d551cb19b1c\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:91e604cc810e8acc9e71aedac5f889852775564824ff256793aec545d3eedefd\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d485fc4783774496efae163751d0bfab45156d5ff0ac545521ed38ee1753822\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7" podUID="42f98018-4b08-482d-b36d-5caaa3b6b522"
Aug 03 16:04:21.290760 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.290659    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9mzt" event={"ID":"61322de4-f04c-4b27-902b-7bbddc29b6b7","Type":"ContainerStarted","Data":"f0e968177030592c927f2ca389aff56ad7399aa06cd424a04b3e692b485fde52"}
Aug 03 16:04:21.299548 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.299515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal" event={"ID":"9d96742d16eb1498950aa8db69ed93ac","Type":"ContainerStarted","Data":"c0a353ee9edff1ab4c58b84d2b4b31de32c494a51dbc62abd12e5105e9b978cd"}
Aug 03 16:04:21.308284 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.308261    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n989x" event={"ID":"09bca23f-ea74-4960-909e-4662dea6789b","Type":"ContainerStarted","Data":"4809170fa5d2a66488c5bf71f9c5c8933ee26897384e848dcd1cdec1049243c1"}
Aug 03 16:04:21.318044 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.318019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"032e96e7394a82bb3f05dccc0430d37f8581774d52883d71838e20f24397d7b2"}
Aug 03 16:04:21.322901 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.322877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lrmsx" event={"ID":"623f60cf-b75a-47d9-bd9d-5d4f954ac74e","Type":"ContainerStarted","Data":"960330361c2ebe2492bd38009584f8368c89f904db19f8972cd1e42fd56ec43b"}
Aug 03 16:04:21.324570 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.324534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerStarted","Data":"8306fae755aaf7d51b99467760b0e93bccb315cde6abab943181d8edfc5c6d64"}
Aug 03 16:04:21.330915 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.330888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jkhtn" event={"ID":"79aebf44-1875-4f4e-a708-c21df0257b1d","Type":"ContainerStarted","Data":"26a4858abfb6d69680ed3f507375d0312bc067322ce889ae4a7f00bbb149b6fc"}
Aug 03 16:04:21.338612 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.338585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-69qfd" event={"ID":"c244ee48-8f52-4a87-92cd-df1c00b2d8ca","Type":"ContainerStarted","Data":"422560cff467fb1cea18476aab3b8b62c77890c2a90aee9031c7e5fcade757e2"}
Aug 03 16:04:21.723365 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.723330    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:21.723539 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.723508    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:21.723539 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.723527    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:21.723539 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.723538    2564 projected.go:196] Error preparing data for projected volume kube-api-access-l2vhk for pod openshift-network-diagnostics/network-check-target-drsdv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:21.723695 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.723589    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk podName:d377dce8-488f-49f1-9e16-9f661e0aba4a nodeName:}" failed. No retries permitted until 2026-08-03 16:04:23.72357297 +0000 UTC m=+5.990388978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l2vhk" (UniqueName: "kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk") pod "network-check-target-drsdv" (UID: "d377dce8-488f-49f1-9e16-9f661e0aba4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:21.824375 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.824308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:21.824560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:21.824395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:21.824560 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.824524    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:21.824671 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.824583    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs podName:9ab2f368-65f9-45e5-912b-c5d207c9ea22 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:23.824566193 +0000 UTC m=+6.091382193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs") pod "network-metrics-daemon-sb5nd" (UID: "9ab2f368-65f9-45e5-912b-c5d207c9ea22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:21.824994 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.824976    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:21.825091 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:21.825025    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret podName:c18f9f7f-3374-4214-b9e8-a98a23f5cc98 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:23.825011091 +0000 UTC m=+6.091827081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret") pod "global-pull-secret-syncer-5nw44" (UID: "c18f9f7f-3374-4214-b9e8-a98a23f5cc98") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:22.134092 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:22.134049    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 15:59:19 +0000 UTC" deadline="2028-01-20 18:22:03.421066291 +0000 UTC"
Aug 03 16:04:22.134092 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:22.134090    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12842h17m41.286980319s"
Aug 03 16:04:22.237910 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:22.237833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:22.238093 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:22.237963    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:22.238589 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:22.238381    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:22.238589 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:22.238486    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:23.238393 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:23.237817    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:23.238393 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.237943    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:23.741561 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:23.741520    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:23.741766 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.741696    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:23.741766 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.741719    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:23.741766 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.741732    2564 projected.go:196] Error preparing data for projected volume kube-api-access-l2vhk for pod openshift-network-diagnostics/network-check-target-drsdv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:23.741944 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.741812    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk podName:d377dce8-488f-49f1-9e16-9f661e0aba4a nodeName:}" failed. No retries permitted until 2026-08-03 16:04:27.741792934 +0000 UTC m=+10.008608942 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l2vhk" (UniqueName: "kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk") pod "network-check-target-drsdv" (UID: "d377dce8-488f-49f1-9e16-9f661e0aba4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:23.843132 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:23.842616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:23.843132 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:23.842684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:23.843132 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.842821    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:23.843132 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.842890    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret podName:c18f9f7f-3374-4214-b9e8-a98a23f5cc98 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:27.842870595 +0000 UTC m=+10.109686605 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret") pod "global-pull-secret-syncer-5nw44" (UID: "c18f9f7f-3374-4214-b9e8-a98a23f5cc98") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:23.843132 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.842935    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:23.843132 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:23.842999    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs podName:9ab2f368-65f9-45e5-912b-c5d207c9ea22 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:27.842981826 +0000 UTC m=+10.109797834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs") pod "network-metrics-daemon-sb5nd" (UID: "9ab2f368-65f9-45e5-912b-c5d207c9ea22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:24.237820 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:24.237777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:24.238007 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:24.237898    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:24.238328 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:24.238302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:24.238445 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:24.238415    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:25.237722 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:25.237687    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:25.237925 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:25.237842    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:26.242304 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:26.242275    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:26.242781 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:26.242420    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:26.242888 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:26.242852    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:26.243045 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:26.243020    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:27.238706 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:27.238219    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:27.238706 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.238360    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:27.777579 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:27.777545    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:27.778082 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.777759    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:27.778082 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.777785    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:27.778082 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.777804    2564 projected.go:196] Error preparing data for projected volume kube-api-access-l2vhk for pod openshift-network-diagnostics/network-check-target-drsdv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:27.778082 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.777869    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk podName:d377dce8-488f-49f1-9e16-9f661e0aba4a nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.777850131 +0000 UTC m=+18.044666116 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l2vhk" (UniqueName: "kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk") pod "network-check-target-drsdv" (UID: "d377dce8-488f-49f1-9e16-9f661e0aba4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:27.878546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:27.878299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:27.878546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:27.878402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:27.878546 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.878475    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:27.878546 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.878514    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:27.878546 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.878547    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret podName:c18f9f7f-3374-4214-b9e8-a98a23f5cc98 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.87852812 +0000 UTC m=+18.145344128 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret") pod "global-pull-secret-syncer-5nw44" (UID: "c18f9f7f-3374-4214-b9e8-a98a23f5cc98") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:27.878991 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:27.878570    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs podName:9ab2f368-65f9-45e5-912b-c5d207c9ea22 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.878559342 +0000 UTC m=+18.145375327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs") pod "network-metrics-daemon-sb5nd" (UID: "9ab2f368-65f9-45e5-912b-c5d207c9ea22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:28.242588 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:28.241948    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:28.242588 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:28.242062    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:28.242588 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:28.242448    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:28.242588 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:28.242550    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:29.238351 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:29.238319    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:29.238829 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:29.238468    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:30.240788 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:30.240754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:30.241228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:30.240806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:30.241228 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:30.240878    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:30.241228 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:30.241000    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:31.237787 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:31.237749    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:31.237969 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:31.237892    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:32.241126 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:32.241092    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:32.241599 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:32.241221    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:32.241599 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:32.241264    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:32.241599 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:32.241374    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:33.237485 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:33.237455    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:33.237653 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:33.237556    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:34.238188 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:34.238151    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:34.238621 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:34.238266    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:34.238621 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:34.238324    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:34.238621 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:34.238446    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:35.238412 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:35.238368    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:35.238871 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.238490    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:35.835479 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:35.835445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:35.835670 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.835639    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:35.835716 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.835671    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:35.835716 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.835687    2564 projected.go:196] Error preparing data for projected volume kube-api-access-l2vhk for pod openshift-network-diagnostics/network-check-target-drsdv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:35.835826 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.835764    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk podName:d377dce8-488f-49f1-9e16-9f661e0aba4a nodeName:}" failed. No retries permitted until 2026-08-03 16:04:51.835744955 +0000 UTC m=+34.102560955 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l2vhk" (UniqueName: "kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk") pod "network-check-target-drsdv" (UID: "d377dce8-488f-49f1-9e16-9f661e0aba4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:35.935840 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:35.935803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:35.936037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:35.935888    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:35.936037 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.935962    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:35.936037 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.935975    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:35.936037 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.936034    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret podName:c18f9f7f-3374-4214-b9e8-a98a23f5cc98 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:51.936017166 +0000 UTC m=+34.202833162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret") pod "global-pull-secret-syncer-5nw44" (UID: "c18f9f7f-3374-4214-b9e8-a98a23f5cc98") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:35.936222 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:35.936053    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs podName:9ab2f368-65f9-45e5-912b-c5d207c9ea22 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:51.936044095 +0000 UTC m=+34.202860081 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs") pod "network-metrics-daemon-sb5nd" (UID: "9ab2f368-65f9-45e5-912b-c5d207c9ea22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:36.241103 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:36.241036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:36.241487 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:36.241162    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:36.241487 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:36.241196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:36.241487 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:36.241281    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:37.237789 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:37.237756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:37.237981 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:37.237923    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:38.238432 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:38.238400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:38.238883 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:38.238521    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:38.238883 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:38.238574    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:38.238883 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:38.238688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:39.237870 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.237511    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:39.237972 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:39.237938    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:39.383179 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.383140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerStarted","Data":"11b381642089c72139afc4a0c665840258e6d7a0389cbbdb0231cf4c5808eff0"}
Aug 03 16:04:39.385454 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.385424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jkhtn" event={"ID":"79aebf44-1875-4f4e-a708-c21df0257b1d","Type":"ContainerStarted","Data":"28ee31e6fa5497560d7efdf2fe6e4cef2a491c4a7674e310be48b33a7bc3fc58"}
Aug 03 16:04:39.387350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.387319    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-69qfd" event={"ID":"c244ee48-8f52-4a87-92cd-df1c00b2d8ca","Type":"ContainerStarted","Data":"aa2a161b3a5c34db8213d45809513f50dae7a4275655b960943819d1ea767a95"}
Aug 03 16:04:39.390110 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.390063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal" event={"ID":"ab1f6e49c95683c2cc3b4fd509ec7a98","Type":"ContainerStarted","Data":"5783b60b2d6ba73042df2e2202aa49c86efaa1b4624bdc8b73d33a9d4d7c1c9c"}
Aug 03 16:04:39.392491 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.392468    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2dtgs" event={"ID":"e16925c5-9e44-4b01-bb5d-fd1172662359","Type":"ContainerStarted","Data":"ae3a5e802506cbf9447579d8974de2d5d31145f548de0450559b65701e818f86"}
Aug 03 16:04:39.394494 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.394470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9mzt" event={"ID":"61322de4-f04c-4b27-902b-7bbddc29b6b7","Type":"ContainerStarted","Data":"18bc46b5565704120d59b8657efabeaacd29f5ba9208d6680cbe44a152393d52"}
Aug 03 16:04:39.396796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.396772    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal" event={"ID":"9d96742d16eb1498950aa8db69ed93ac","Type":"ContainerStarted","Data":"82e81a6fac59907faeab39d753267554b7c1107b2ee3c5477043af19d3b784b2"}
Aug 03 16:04:39.398691 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.398670    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n989x" event={"ID":"09bca23f-ea74-4960-909e-4662dea6789b","Type":"ContainerStarted","Data":"1bda9f95a4894b80779e70d8e9196b7f87faac2d4442cbbda209e3451a65fba6"}
Aug 03 16:04:39.402178 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.402161    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:04:39.402804 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.402775    2564 generic.go:356] "Generic (PLEG): container finished" podID="1fe44827-6e6a-4c1e-a105-a6734ddbb548" containerID="70bee9d3102c1eb91dd5ccfa611260edf73f78ed731ed3bd5bf4cc67e6245c03" exitCode=1
Aug 03 16:04:39.402875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.402832    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"75ed45d5e77ced9781d40e2780ee8b1c9c454af8611093bebdbd5978ea871a44"}
Aug 03 16:04:39.405296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.402991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"daa25285c7f78fccc596c6cf4e395aa84e66f4e241b10480b7fccff08d15060b"}
Aug 03 16:04:39.405296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.403079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"de72be5326cd4cd2a9bbef1c7758c801bb6f707ba314f6ad7bcf7f2dc4265622"}
Aug 03 16:04:39.405296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.403142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"921c0114b5e77a1a550e1f213557a7d580319323a5d8da204fe64d076f61ba26"}
Aug 03 16:04:39.405296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.403206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerDied","Data":"70bee9d3102c1eb91dd5ccfa611260edf73f78ed731ed3bd5bf4cc67e6245c03"}
Aug 03 16:04:39.405296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.403273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"1ae038186d9b646cf7da7a0d89a8047d6e5962038ea879ad00fbd0db68e4421e"}
Aug 03 16:04:39.423285 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.423221    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-110.ec2.internal" podStartSLOduration=20.42320196 podStartE2EDuration="20.42320196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:39.423066125 +0000 UTC m=+21.689882132" watchObservedRunningTime="2026-08-03 16:04:39.42320196 +0000 UTC m=+21.690017973"
Aug 03 16:04:39.440079 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.440040    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-n989x" podStartSLOduration=4.172631232 podStartE2EDuration="22.440026535s" totalImagesPullingTime="18.267395303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:39.439803242 +0000 UTC m=+21.706619249" watchObservedRunningTime="2026-08-03 16:04:39.440026535 +0000 UTC m=+21.706842542"
Aug 03 16:04:39.486350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.486194    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jkhtn" podStartSLOduration=3.26308341 podStartE2EDuration="21.486174598s" totalImagesPullingTime="18.223091188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:39.458844196 +0000 UTC m=+21.725660204" watchObservedRunningTime="2026-08-03 16:04:39.486174598 +0000 UTC m=+21.752990606"
Aug 03 16:04:39.500529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.500480    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-69qfd" podStartSLOduration=12.271606078 podStartE2EDuration="22.500466287s" totalImagesPullingTime="10.228860209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:39.500263429 +0000 UTC m=+21.767079434" watchObservedRunningTime="2026-08-03 16:04:39.500466287 +0000 UTC m=+21.767282295"
Aug 03 16:04:39.517825 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.517717    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g9mzt" podStartSLOduration=3.962814412 podStartE2EDuration="22.517697835s" totalImagesPullingTime="18.554883423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:39.516267303 +0000 UTC m=+21.783083310" watchObservedRunningTime="2026-08-03 16:04:39.517697835 +0000 UTC m=+21.784513846"
Aug 03 16:04:39.531329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:39.531283    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2dtgs" podStartSLOduration=4.278954599 podStartE2EDuration="22.531264466s" totalImagesPullingTime="18.252309867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:39.530441907 +0000 UTC m=+21.797257916" watchObservedRunningTime="2026-08-03 16:04:39.531264466 +0000 UTC m=+21.798080478"
Aug 03 16:04:40.238173 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.238149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:40.238282 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.238149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:40.238282 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:40.238255    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:40.238378 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:40.238320    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:40.406675 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.406472    2564 generic.go:356] "Generic (PLEG): container finished" podID="9d96742d16eb1498950aa8db69ed93ac" containerID="82e81a6fac59907faeab39d753267554b7c1107b2ee3c5477043af19d3b784b2" exitCode=0
Aug 03 16:04:40.407150 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.406554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal" event={"ID":"9d96742d16eb1498950aa8db69ed93ac","Type":"ContainerDied","Data":"82e81a6fac59907faeab39d753267554b7c1107b2ee3c5477043af19d3b784b2"}
Aug 03 16:04:40.407150 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.406787    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal" event={"ID":"9d96742d16eb1498950aa8db69ed93ac","Type":"ContainerStarted","Data":"0cfb864af1a4a9562b2ecaa16bee71580a4090edcbc4f15f7aca5dfe7bc314d2"}
Aug 03 16:04:40.408088 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.408063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lrmsx" event={"ID":"623f60cf-b75a-47d9-bd9d-5d4f954ac74e","Type":"ContainerStarted","Data":"37452714d8f9ddde6f5c13995a94d483ad06ab1c08be6908b24f44c149585b2d"}
Aug 03 16:04:40.409416 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.409386    2564 generic.go:356] "Generic (PLEG): container finished" podID="b1474a27-5420-4708-9008-10067c3fddca" containerID="11b381642089c72139afc4a0c665840258e6d7a0389cbbdb0231cf4c5808eff0" exitCode=0
Aug 03 16:04:40.409505 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.409483    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerDied","Data":"11b381642089c72139afc4a0c665840258e6d7a0389cbbdb0231cf4c5808eff0"}
Aug 03 16:04:40.423456 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.423420    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-110.ec2.internal" podStartSLOduration=21.423409395 podStartE2EDuration="21.423409395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:40.421949715 +0000 UTC m=+22.688765721" watchObservedRunningTime="2026-08-03 16:04:40.423409395 +0000 UTC m=+22.690225401"
Aug 03 16:04:40.436290 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:40.436255    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lrmsx" podStartSLOduration=4.2541122510000005 podStartE2EDuration="22.436245363s" totalImagesPullingTime="18.182133112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:40.436004722 +0000 UTC m=+22.702820740" watchObservedRunningTime="2026-08-03 16:04:40.436245363 +0000 UTC m=+22.703061402"
Aug 03 16:04:41.237468 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:41.237441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:41.237694 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:41.237533    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:41.413199 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:41.413161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7" event={"ID":"42f98018-4b08-482d-b36d-5caaa3b6b522","Type":"ContainerStarted","Data":"c36c4813b11fde8a99e838b709c5151fa561ae69946554b62617cdbc8f625e18"}
Aug 03 16:04:42.174429 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:42.174408    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 03 16:04:42.238424 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:42.238390    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:42.238578 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:42.238433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:42.238578 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:42.238538    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:42.238693 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:42.238616    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:42.417252 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:42.417173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7" event={"ID":"42f98018-4b08-482d-b36d-5caaa3b6b522","Type":"ContainerStarted","Data":"f5c2a7a030f74cbf21b660b4103588175c59e6cb7c6cef86087fd73b7d93c3a3"}
Aug 03 16:04:42.420365 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:42.420345    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:04:42.420676 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:42.420659    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"d064baa90c4637e94ebb5ca7974b9ffbb13f58b78f8c7aa5f30476ca22567867"}
Aug 03 16:04:43.168327 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:43.168237    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-03T16:04:42.174425973Z","UUID":"a430ea43-3250-4662-831a-e22d4c963164","Handler":null,"Name":"","Endpoint":""}
Aug 03 16:04:43.170014 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:43.169992    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 03 16:04:43.170014 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:43.170020    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 03 16:04:43.237781 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:43.237753    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:43.237940 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:43.237859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:44.121803 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.121773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:44.122380 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.122364    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:44.238599 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.238518    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:44.238771 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:44.238643    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:44.238771 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.238710    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:44.238895 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:44.238853    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:44.428266 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.428225    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7" event={"ID":"42f98018-4b08-482d-b36d-5caaa3b6b522","Type":"ContainerStarted","Data":"816a58da52bb197dd5cc60ec2caf93ecdcc4fe029181b516eb06d71abb4bc90c"}
Aug 03 16:04:44.432525 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.432504    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:04:44.433007 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.432981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"dfba605033b801146f61c549233d9fc6453ed26f6ea9d764c241af807f790398"}
Aug 03 16:04:44.433478 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.433462    2564 scope.go:117] "RemoveContainer" containerID="70bee9d3102c1eb91dd5ccfa611260edf73f78ed731ed3bd5bf4cc67e6245c03"
Aug 03 16:04:44.447762 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:44.447699    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jd5t7" podStartSLOduration=3.013607733 podStartE2EDuration="26.447682456s" totalImagesPullingTime="23.434074723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-08-03 16:04:44.447234902 +0000 UTC m=+26.714050911" watchObservedRunningTime="2026-08-03 16:04:44.447682456 +0000 UTC m=+26.714498465"
Aug 03 16:04:45.238112 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.238087    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:45.238633 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:45.238195    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:45.442104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.442017    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:04:45.442728 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.442697    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" event={"ID":"1fe44827-6e6a-4c1e-a105-a6734ddbb548","Type":"ContainerStarted","Data":"c7bacb19ce373d13b699ab30f9191f3e624de4c20a5b417738705484681d6763"}
Aug 03 16:04:45.442956 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.442936    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 03 16:04:45.443233 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.443209    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:45.443365 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.443290    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:45.464831 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.464800    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:45.464969 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.464953    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:45.476338 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.476288    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd" podStartSLOduration=9.279892261 podStartE2EDuration="27.476269071s" totalImagesPullingTime="18.19637681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:45.474552129 +0000 UTC m=+27.741368139" watchObservedRunningTime="2026-08-03 16:04:45.476269071 +0000 UTC m=+27.743085080"
Aug 03 16:04:45.682419 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.682281    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5nw44"]
Aug 03 16:04:45.682602 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.682453    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:45.682602 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:45.682561    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:45.685225 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.685188    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-drsdv"]
Aug 03 16:04:45.685365 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.685294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:45.685431 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:45.685376    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:45.686233 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.685975    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sb5nd"]
Aug 03 16:04:45.686233 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.686077    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:45.686233 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:45.686192    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:45.865430 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:45.865386    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:04:46.704347 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:46.704160    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:46.704717 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:46.704432    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 03 16:04:46.704851 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:46.704833    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jkhtn"
Aug 03 16:04:47.238503 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:47.238427    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:47.238656 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:47.238525    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:47.238656 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:47.238537    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:47.238656 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:47.238547    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:47.238656 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:47.238606    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:47.238910 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:47.238692    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:47.449020 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:47.448989    2564 generic.go:356] "Generic (PLEG): container finished" podID="b1474a27-5420-4708-9008-10067c3fddca" containerID="afed9602e79c8235040d25d39b60d8a6276df191448c66c6958b13a4df52d7c8" exitCode=0
Aug 03 16:04:47.449152 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:47.449045    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerDied","Data":"afed9602e79c8235040d25d39b60d8a6276df191448c66c6958b13a4df52d7c8"}
Aug 03 16:04:49.238159 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:49.238119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:49.238490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:49.238119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:49.238490 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:49.238219    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:49.238490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:49.238119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:49.238490 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:49.238297    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:49.238490 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:49.238361    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:49.455041 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:49.455011    2564 generic.go:356] "Generic (PLEG): container finished" podID="b1474a27-5420-4708-9008-10067c3fddca" containerID="18fcddd6c27860cf494974cf669186337eca9b96cbacb6627ecd8f75e8ca9b67" exitCode=0
Aug 03 16:04:49.455183 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:49.455064    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerDied","Data":"18fcddd6c27860cf494974cf669186337eca9b96cbacb6627ecd8f75e8ca9b67"}
Aug 03 16:04:50.555337 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:50.555265    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2dtgs_e16925c5-9e44-4b01-bb5d-fd1172662359/dns-node-resolver/0.log"
Aug 03 16:04:51.237946 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.237915    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:51.238119 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.238006    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:51.238119 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.238013    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:51.238119 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.238010    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:51.238119 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.238106    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:51.238315 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.238207    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:51.462336 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.462302    2564 generic.go:356] "Generic (PLEG): container finished" podID="b1474a27-5420-4708-9008-10067c3fddca" containerID="f25d81f7f96029d71ff1273a45a5f444b9a30a34d8a7a83f67fa381c360b72e0" exitCode=0
Aug 03 16:04:51.462486 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.462358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerDied","Data":"f25d81f7f96029d71ff1273a45a5f444b9a30a34d8a7a83f67fa381c360b72e0"}
Aug 03 16:04:51.728774 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.728732    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-69qfd_c244ee48-8f52-4a87-92cd-df1c00b2d8ca/node-ca/0.log"
Aug 03 16:04:51.858233 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.858200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:51.858390 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.858327    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:51.858390 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.858342    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:51.858390 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.858351    2564 projected.go:196] Error preparing data for projected volume kube-api-access-l2vhk for pod openshift-network-diagnostics/network-check-target-drsdv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:51.858483 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.858401    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk podName:d377dce8-488f-49f1-9e16-9f661e0aba4a nodeName:}" failed. No retries permitted until 2026-08-03 16:05:23.858383897 +0000 UTC m=+66.125199882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l2vhk" (UniqueName: "kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk") pod "network-check-target-drsdv" (UID: "d377dce8-488f-49f1-9e16-9f661e0aba4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:51.959297 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.959265    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:51.959444 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:51.959325    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:51.959444 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.959406    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:51.959516 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.959444    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:51.959516 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.959466    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs podName:9ab2f368-65f9-45e5-912b-c5d207c9ea22 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:23.959449079 +0000 UTC m=+66.226265064 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs") pod "network-metrics-daemon-sb5nd" (UID: "9ab2f368-65f9-45e5-912b-c5d207c9ea22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:51.959516 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:51.959489    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret podName:c18f9f7f-3374-4214-b9e8-a98a23f5cc98 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:23.959475701 +0000 UTC m=+66.226291707 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret") pod "global-pull-secret-syncer-5nw44" (UID: "c18f9f7f-3374-4214-b9e8-a98a23f5cc98") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:53.237930 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:53.237897    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:53.238409 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:53.237936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:53.238409 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:53.237942    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:53.238409 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:53.238013    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:53.238409 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:53.238095    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:53.238409 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:53.238187    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:55.237856 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:55.237628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:55.238203 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:55.237649    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:55.238203 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:55.237945    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:55.238203 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:55.238007    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:55.238203 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:55.237707    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:55.238203 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:55.238098    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:57.237614 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:57.237572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:57.238185 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:57.237677    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:57.238185 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:57.237681    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:57.238185 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:57.237714    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:57.238185 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:57.237833    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:57.238185 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:57.237946    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:57.476801 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:57.476763    2564 generic.go:356] "Generic (PLEG): container finished" podID="b1474a27-5420-4708-9008-10067c3fddca" containerID="c315b9e29e1ea5d1dbd956fc1a8ff484c5fcbf9f580b779d325da61bb7108158" exitCode=0
Aug 03 16:04:57.476954 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:57.476827    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerDied","Data":"c315b9e29e1ea5d1dbd956fc1a8ff484c5fcbf9f580b779d325da61bb7108158"}
Aug 03 16:04:58.481982 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:58.481949    2564 generic.go:356] "Generic (PLEG): container finished" podID="b1474a27-5420-4708-9008-10067c3fddca" containerID="8077a6ba27b9fff64c5a2a9cb2cf169719c1e79bee494c73e2b9101e7e9801a9" exitCode=0
Aug 03 16:04:58.482351 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:58.481991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerDied","Data":"8077a6ba27b9fff64c5a2a9cb2cf169719c1e79bee494c73e2b9101e7e9801a9"}
Aug 03 16:04:59.237538 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:59.237503    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:04:59.237538 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:59.237535    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:04:59.237767 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:59.237503    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:04:59.237767 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:59.237605    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:04:59.237767 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:59.237665    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:04:59.237870 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:04:59.237765    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:04:59.487596 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:59.487521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" event={"ID":"b1474a27-5420-4708-9008-10067c3fddca","Type":"ContainerStarted","Data":"8bf3c4c45903d48443f4965f2107f898c033793d2224870c2affc318e2380cc1"}
Aug 03 16:04:59.510109 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:04:59.510068    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z5qrk" podStartSLOduration=9.627060977 podStartE2EDuration="41.510053942s" totalImagesPullingTime="31.882992965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:59.509956283 +0000 UTC m=+41.776772290" watchObservedRunningTime="2026-08-03 16:04:59.510053942 +0000 UTC m=+41.776869948"
Aug 03 16:05:01.237819 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:01.237787    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:01.238288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:01.237791    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:01.238288 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:01.237905    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:05:01.238288 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:01.237962    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:05:01.238288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:01.237791    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:01.238288 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:01.238043    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:05:03.237503 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:03.237463    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:03.237503 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:03.237488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:03.237988 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:03.237623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:03.237988 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:03.237632    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:05:03.237988 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:03.237688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:05:03.237988 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:03.237793    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:05:05.238373 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:05.238341    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:05.238373 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:05.238369    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:05.238830 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:05.238455    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:05:05.238830 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:05.238487    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:05:05.238830 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:05.238540    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:05.238830 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:05.238599    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:05:07.237690 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:07.237652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:07.238183 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:07.237701    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:07.238183 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:07.237773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:07.238183 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:07.237791    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:05:07.238183 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:07.237853    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:05:07.238183 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:07.237898    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:05:09.237810 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:09.237773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:09.237810 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:09.237795    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:09.238374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:09.237774    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:09.238374 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:09.237903    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:05:09.238374 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:09.238015    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:05:09.238374 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:09.238079    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:05:11.237956 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:11.237925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:11.238433 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:11.238013    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:11.238433 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:11.238018    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:05:11.238433 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:11.238036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:11.238433 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:11.238103    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:05:11.238433 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:11.238167    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:05:13.238288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.238255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:13.238288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.238268    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:13.238716 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.238294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:13.238716 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:13.238366    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sb5nd" podUID="9ab2f368-65f9-45e5-912b-c5d207c9ea22"
Aug 03 16:05:13.238716 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:13.238414    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-drsdv" podUID="d377dce8-488f-49f1-9e16-9f661e0aba4a"
Aug 03 16:05:13.238716 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:13.238493    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nw44" podUID="c18f9f7f-3374-4214-b9e8-a98a23f5cc98"
Aug 03 16:05:13.529843 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.529724    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-110.ec2.internal" event="NodeReady"
Aug 03 16:05:13.529946 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.529895    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 03 16:05:13.564863 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.564830    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58568488cc-mxl9t"]
Aug 03 16:05:13.568333 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.568319    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.571070 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.571048    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 03 16:05:13.571170 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.571048    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 03 16:05:13.571564 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.571547    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xchfl\""
Aug 03 16:05:13.571801 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.571787    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 03 16:05:13.578822 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.578804    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 03 16:05:13.582841 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.582824    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58568488cc-mxl9t"]
Aug 03 16:05:13.586371 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.586341    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5zd6c"]
Aug 03 16:05:13.589783 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.589767    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mp7kr"]
Aug 03 16:05:13.589900 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.589885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.593044 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.593030    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:13.599529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.599392    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 03 16:05:13.599529 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:13.599389    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Aug 03 16:05:13.599529 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:13.599389    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Aug 03 16:05:13.599529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.599461    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 03 16:05:13.599529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.599471    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 03 16:05:13.599529 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:13.599481    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-dkjck\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dkjck\"" type="*v1.Secret"
Aug 03 16:05:13.599898 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.599574    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vz785\""
Aug 03 16:05:13.604870 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.604853    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zd6c"]
Aug 03 16:05:13.608110 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.608087    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mp7kr"]
Aug 03 16:05:13.625853 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.625829    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-metrics-tls\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.625949 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.625859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22pfh\" (UniqueName: \"kubernetes.io/projected/13857d42-267a-40b4-959a-85e751549e0b-kube-api-access-22pfh\") pod \"ingress-canary-mp7kr\" (UID: \"13857d42-267a-40b4-959a-85e751549e0b\") " pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:13.625949 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.625888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-config-volume\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.625949 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.625933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f52dae94-350b-4c29-86a5-c8cd5440f5c7-ca-trust-extracted\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.626101 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.625956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f52dae94-350b-4c29-86a5-c8cd5440f5c7-trusted-ca\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.626101 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.625976    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f52dae94-350b-4c29-86a5-c8cd5440f5c7-installation-pull-secrets\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.626101 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-registry-tls\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.626101 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqs65\" (UniqueName: \"kubernetes.io/projected/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-kube-api-access-zqs65\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.626243 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626139    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-bound-sa-token\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.626243 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-tmp-dir\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.626243 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f52dae94-350b-4c29-86a5-c8cd5440f5c7-registry-certificates\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.626243 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcx6j\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-kube-api-access-qcx6j\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.626243 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626218    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13857d42-267a-40b4-959a-85e751549e0b-cert\") pod \"ingress-canary-mp7kr\" (UID: \"13857d42-267a-40b4-959a-85e751549e0b\") " pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:13.626243 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.626234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f52dae94-350b-4c29-86a5-c8cd5440f5c7-image-registry-private-configuration\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.683042 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.683016    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wg65w"]
Aug 03 16:05:13.686395 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.686381    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.689099 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.689081    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 03 16:05:13.689356 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.689341    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 03 16:05:13.689413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.689358    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-grgrw\""
Aug 03 16:05:13.689464 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.689413    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 03 16:05:13.689464 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.689414    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 03 16:05:13.694684 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.694659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wg65w"]
Aug 03 16:05:13.726773 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-bound-sa-token\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.726880 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-tmp-dir\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.726880 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f52dae94-350b-4c29-86a5-c8cd5440f5c7-registry-certificates\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.726880 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcx6j\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-kube-api-access-qcx6j\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.727035 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726890    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13857d42-267a-40b4-959a-85e751549e0b-cert\") pod \"ingress-canary-mp7kr\" (UID: \"13857d42-267a-40b4-959a-85e751549e0b\") " pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:13.727035 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f52dae94-350b-4c29-86a5-c8cd5440f5c7-image-registry-private-configuration\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.727035 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-metrics-tls\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.727035 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726974    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45cbc0e2-f246-44a8-a38b-ed39d24ac032-crio-socket\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.727035 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.726998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45cbc0e2-f246-44a8-a38b-ed39d24ac032-data-volume\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22pfh\" (UniqueName: \"kubernetes.io/projected/13857d42-267a-40b4-959a-85e751549e0b-kube-api-access-22pfh\") pod \"ingress-canary-mp7kr\" (UID: \"13857d42-267a-40b4-959a-85e751549e0b\") " pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-tmp-dir\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45cbc0e2-f246-44a8-a38b-ed39d24ac032-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-config-volume\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727174    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f52dae94-350b-4c29-86a5-c8cd5440f5c7-ca-trust-extracted\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727193    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f52dae94-350b-4c29-86a5-c8cd5440f5c7-trusted-ca\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f52dae94-350b-4c29-86a5-c8cd5440f5c7-installation-pull-secrets\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45cbc0e2-f246-44a8-a38b-ed39d24ac032-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.727268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlnf2\" (UniqueName: \"kubernetes.io/projected/45cbc0e2-f246-44a8-a38b-ed39d24ac032-kube-api-access-vlnf2\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.727662 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-registry-tls\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.727662 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727348    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqs65\" (UniqueName: \"kubernetes.io/projected/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-kube-api-access-zqs65\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.727662 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.727643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-config-volume\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.728308 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.728058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f52dae94-350b-4c29-86a5-c8cd5440f5c7-registry-certificates\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.728418 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.728392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f52dae94-350b-4c29-86a5-c8cd5440f5c7-ca-trust-extracted\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.728418 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.728323    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f52dae94-350b-4c29-86a5-c8cd5440f5c7-trusted-ca\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.731243 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.731220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f52dae94-350b-4c29-86a5-c8cd5440f5c7-image-registry-private-configuration\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.731323 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.731258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-metrics-tls\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.731323 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.731298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f52dae94-350b-4c29-86a5-c8cd5440f5c7-installation-pull-secrets\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.731323 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.731305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-registry-tls\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.731323 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.731319    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13857d42-267a-40b4-959a-85e751549e0b-cert\") pod \"ingress-canary-mp7kr\" (UID: \"13857d42-267a-40b4-959a-85e751549e0b\") " pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:13.735362 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.735343    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqs65\" (UniqueName: \"kubernetes.io/projected/6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc-kube-api-access-zqs65\") pod \"dns-default-5zd6c\" (UID: \"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc\") " pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.736000 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.735979    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-bound-sa-token\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.736251 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.736232    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcx6j\" (UniqueName: \"kubernetes.io/projected/f52dae94-350b-4c29-86a5-c8cd5440f5c7-kube-api-access-qcx6j\") pod \"image-registry-58568488cc-mxl9t\" (UID: \"f52dae94-350b-4c29-86a5-c8cd5440f5c7\") " pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.828541 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.828511    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45cbc0e2-f246-44a8-a38b-ed39d24ac032-crio-socket\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.828689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.828544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45cbc0e2-f246-44a8-a38b-ed39d24ac032-data-volume\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.828689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.828583    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45cbc0e2-f246-44a8-a38b-ed39d24ac032-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.828689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.828627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45cbc0e2-f246-44a8-a38b-ed39d24ac032-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.828689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.828631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45cbc0e2-f246-44a8-a38b-ed39d24ac032-crio-socket\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.828689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.828651    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlnf2\" (UniqueName: \"kubernetes.io/projected/45cbc0e2-f246-44a8-a38b-ed39d24ac032-kube-api-access-vlnf2\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.828959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.828906    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45cbc0e2-f246-44a8-a38b-ed39d24ac032-data-volume\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.829192 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.829172    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45cbc0e2-f246-44a8-a38b-ed39d24ac032-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.830775 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.830757    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45cbc0e2-f246-44a8-a38b-ed39d24ac032-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.837159 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.837138    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlnf2\" (UniqueName: \"kubernetes.io/projected/45cbc0e2-f246-44a8-a38b-ed39d24ac032-kube-api-access-vlnf2\") pod \"insights-runtime-extractor-wg65w\" (UID: \"45cbc0e2-f246-44a8-a38b-ed39d24ac032\") " pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:13.877612 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.877586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:13.898336 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.898318    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:13.995128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:13.995104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wg65w"
Aug 03 16:05:14.023103 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.023075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58568488cc-mxl9t"]
Aug 03 16:05:14.026634 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:14.026590    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52dae94_350b_4c29_86a5_c8cd5440f5c7.slice/crio-85e864a92afba349fa91b9556d6df8233d78a3ff37dd612a95cf341cc9aef32c WatchSource:0}: Error finding container 85e864a92afba349fa91b9556d6df8233d78a3ff37dd612a95cf341cc9aef32c: Status 404 returned error can't find the container with id 85e864a92afba349fa91b9556d6df8233d78a3ff37dd612a95cf341cc9aef32c
Aug 03 16:05:14.037221 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.036869    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zd6c"]
Aug 03 16:05:14.039950 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:14.039926    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fbcfd14_0820_4d90_a3a8_9a9d62b5f1fc.slice/crio-6964e3d77ef867a482cfd024f477d30e6d7a4dbd4b695f69450f37eed37be9a7 WatchSource:0}: Error finding container 6964e3d77ef867a482cfd024f477d30e6d7a4dbd4b695f69450f37eed37be9a7: Status 404 returned error can't find the container with id 6964e3d77ef867a482cfd024f477d30e6d7a4dbd4b695f69450f37eed37be9a7
Aug 03 16:05:14.126359 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.126332    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wg65w"]
Aug 03 16:05:14.129324 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:14.129296    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45cbc0e2_f246_44a8_a38b_ed39d24ac032.slice/crio-eca0e682106d93a97a528ba9354bbfc704aa1aa474ffd745ba9f1fd0c203eb50 WatchSource:0}: Error finding container eca0e682106d93a97a528ba9354bbfc704aa1aa474ffd745ba9f1fd0c203eb50: Status 404 returned error can't find the container with id eca0e682106d93a97a528ba9354bbfc704aa1aa474ffd745ba9f1fd0c203eb50
Aug 03 16:05:14.502133 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.502060    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dkjck\""
Aug 03 16:05:14.520185 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.520150    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zd6c" event={"ID":"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc","Type":"ContainerStarted","Data":"6964e3d77ef867a482cfd024f477d30e6d7a4dbd4b695f69450f37eed37be9a7"}
Aug 03 16:05:14.521539 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.521513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58568488cc-mxl9t" event={"ID":"f52dae94-350b-4c29-86a5-c8cd5440f5c7","Type":"ContainerStarted","Data":"3fbf928854d2ef1ef574478f63ebd5747bf5e5aead4caa759b4fd5e7cc22f494"}
Aug 03 16:05:14.521677 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.521557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58568488cc-mxl9t" event={"ID":"f52dae94-350b-4c29-86a5-c8cd5440f5c7","Type":"ContainerStarted","Data":"85e864a92afba349fa91b9556d6df8233d78a3ff37dd612a95cf341cc9aef32c"}
Aug 03 16:05:14.521677 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.521581    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:14.522841 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.522822    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wg65w" event={"ID":"45cbc0e2-f246-44a8-a38b-ed39d24ac032","Type":"ContainerStarted","Data":"f1c827bb24e6f1136469d9db78a90d8235234f481ba56ebf091b70d308920009"}
Aug 03 16:05:14.522939 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.522857    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wg65w" event={"ID":"45cbc0e2-f246-44a8-a38b-ed39d24ac032","Type":"ContainerStarted","Data":"eca0e682106d93a97a528ba9354bbfc704aa1aa474ffd745ba9f1fd0c203eb50"}
Aug 03 16:05:14.552135 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.552078    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58568488cc-mxl9t" podStartSLOduration=1.552063062 podStartE2EDuration="1.552063062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:14.55109263 +0000 UTC m=+56.817908639" watchObservedRunningTime="2026-08-03 16:05:14.552063062 +0000 UTC m=+56.818879068"
Aug 03 16:05:14.733478 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:14.733435    2564 projected.go:291] Couldn't get configMap openshift-ingress-canary/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Aug 03 16:05:14.897976 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:14.896887    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 03 16:05:14.904177 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:14.904152    2564 projected.go:196] Error preparing data for projected volume kube-api-access-22pfh for pod openshift-ingress-canary/ingress-canary-mp7kr: failed to sync configmap cache: timed out waiting for the condition
Aug 03 16:05:14.904311 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:14.904232    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13857d42-267a-40b4-959a-85e751549e0b-kube-api-access-22pfh podName:13857d42-267a-40b4-959a-85e751549e0b nodeName:}" failed. No retries permitted until 2026-08-03 16:05:15.40421388 +0000 UTC m=+57.671029883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-22pfh" (UniqueName: "kubernetes.io/projected/13857d42-267a-40b4-959a-85e751549e0b-kube-api-access-22pfh") pod "ingress-canary-mp7kr" (UID: "13857d42-267a-40b4-959a-85e751549e0b") : failed to sync configmap cache: timed out waiting for the condition
Aug 03 16:05:15.127867 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.127835    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 03 16:05:15.238582 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.238503    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:15.238761 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.238503    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:15.238761 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.238503    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:15.244374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.244345    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 03 16:05:15.244511 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.244405    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 03 16:05:15.244511 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.244345    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xrdkc\""
Aug 03 16:05:15.244973 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.244750    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 03 16:05:15.244973 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.244824    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzpf\""
Aug 03 16:05:15.244973 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.244886    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 03 16:05:15.442090 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.442058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22pfh\" (UniqueName: \"kubernetes.io/projected/13857d42-267a-40b4-959a-85e751549e0b-kube-api-access-22pfh\") pod \"ingress-canary-mp7kr\" (UID: \"13857d42-267a-40b4-959a-85e751549e0b\") " pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:15.444981 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.444954    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22pfh\" (UniqueName: \"kubernetes.io/projected/13857d42-267a-40b4-959a-85e751549e0b-kube-api-access-22pfh\") pod \"ingress-canary-mp7kr\" (UID: \"13857d42-267a-40b4-959a-85e751549e0b\") " pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:15.528609 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.528531    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wg65w" event={"ID":"45cbc0e2-f246-44a8-a38b-ed39d24ac032","Type":"ContainerStarted","Data":"e9c6d7215ee1450fb9be1111dd71eeb289de4856788924a2425b6bbd90c19269"}
Aug 03 16:05:15.703014 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:15.702979    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mp7kr"
Aug 03 16:05:16.534502 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:16.534468    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wg65w" event={"ID":"45cbc0e2-f246-44a8-a38b-ed39d24ac032","Type":"ContainerStarted","Data":"6ba9fdc8be5a8e34e9f39178e52deb417ca50541a756c819810011c5c85fa5d3"}
Aug 03 16:05:16.536026 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:16.536000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zd6c" event={"ID":"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc","Type":"ContainerStarted","Data":"bf63864eb704c8da30f524478ffd545188b397e988eb4c117bb6aa5c7c86e73f"}
Aug 03 16:05:16.545693 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:16.545506    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mp7kr"]
Aug 03 16:05:16.548074 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:16.548046    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13857d42_267a_40b4_959a_85e751549e0b.slice/crio-580e4241f7bd6e94d705d59bb524baf4a30594b37633267e466da23b5b05a816 WatchSource:0}: Error finding container 580e4241f7bd6e94d705d59bb524baf4a30594b37633267e466da23b5b05a816: Status 404 returned error can't find the container with id 580e4241f7bd6e94d705d59bb524baf4a30594b37633267e466da23b5b05a816
Aug 03 16:05:16.553410 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:16.553363    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wg65w" podStartSLOduration=1.386361011 podStartE2EDuration="3.55334396s" totalImagesPullingTime="2.166982949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:16.552909708 +0000 UTC m=+58.819725720" watchObservedRunningTime="2026-08-03 16:05:16.55334396 +0000 UTC m=+58.820159968"
Aug 03 16:05:17.468502 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:17.468476    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mprtd"
Aug 03 16:05:17.541910 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:17.541865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zd6c" event={"ID":"6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc","Type":"ContainerStarted","Data":"d85ba071e510e584d5db262ddc3bab715159c15e82a71175678ad4c915d273ef"}
Aug 03 16:05:17.542341 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:17.541982    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:17.543232 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:17.543206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mp7kr" event={"ID":"13857d42-267a-40b4-959a-85e751549e0b","Type":"ContainerStarted","Data":"580e4241f7bd6e94d705d59bb524baf4a30594b37633267e466da23b5b05a816"}
Aug 03 16:05:17.559389 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:17.559351    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5zd6c" podStartSLOduration=2.184089918 podStartE2EDuration="4.55933962s" totalImagesPullingTime="2.375249702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:17.55880568 +0000 UTC m=+59.825621688" watchObservedRunningTime="2026-08-03 16:05:17.55933962 +0000 UTC m=+59.826155628"
Aug 03 16:05:19.550700 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.550660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mp7kr" event={"ID":"13857d42-267a-40b4-959a-85e751549e0b","Type":"ContainerStarted","Data":"866b74934f008f36ed87002a2d89a2aa6ee6ac956a62dcecf44c41b52af86b2d"}
Aug 03 16:05:19.569300 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.569250    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mp7kr" podStartSLOduration=4.378042264 podStartE2EDuration="6.569235121s" totalImagesPullingTime="2.191192857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:19.568852255 +0000 UTC m=+61.835668273" watchObservedRunningTime="2026-08-03 16:05:19.569235121 +0000 UTC m=+61.836051170"
Aug 03 16:05:19.837454 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.837398    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-65jb7"]
Aug 03 16:05:19.840147 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.840127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:19.843176 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.843153    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Aug 03 16:05:19.843321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.843183    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 03 16:05:19.843321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.843189    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Aug 03 16:05:19.843321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.843286    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ncc46\""
Aug 03 16:05:19.843500 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.843483    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 03 16:05:19.843548 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.843535    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 03 16:05:19.853272 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.853245    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-65jb7"]
Aug 03 16:05:19.976987 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.976955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbdv4\" (UniqueName: \"kubernetes.io/projected/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-kube-api-access-tbdv4\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:19.977158 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.977014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:19.977158 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.977072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:19.977158 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:19.977090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.077905 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.077865    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbdv4\" (UniqueName: \"kubernetes.io/projected/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-kube-api-access-tbdv4\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.078037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.077932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.078037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.077964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.078037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.077980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.078131 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:20.078096    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Aug 03 16:05:20.078181 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:20.078168    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-tls podName:daca8c9e-b40e-4c48-98ce-d3c3dc6d692e nodeName:}" failed. No retries permitted until 2026-08-03 16:05:20.578150672 +0000 UTC m=+62.844966658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-tls") pod "prometheus-operator-758d7cb874-65jb7" (UID: "daca8c9e-b40e-4c48-98ce-d3c3dc6d692e") : secret "prometheus-operator-tls" not found
Aug 03 16:05:20.078726 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.078707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.080439 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.080418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.086890 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.086870    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbdv4\" (UniqueName: \"kubernetes.io/projected/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-kube-api-access-tbdv4\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.581626 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.581588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.583987 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.583962    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/daca8c9e-b40e-4c48-98ce-d3c3dc6d692e-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-65jb7\" (UID: \"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.749546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.749496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7"
Aug 03 16:05:20.866230 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:20.866121    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-65jb7"]
Aug 03 16:05:20.868201 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:20.868173    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaca8c9e_b40e_4c48_98ce_d3c3dc6d692e.slice/crio-b20a77dac495afe2fe31d4b57a249b2f4c82f6e205e7483ae20df1d1d83d9362 WatchSource:0}: Error finding container b20a77dac495afe2fe31d4b57a249b2f4c82f6e205e7483ae20df1d1d83d9362: Status 404 returned error can't find the container with id b20a77dac495afe2fe31d4b57a249b2f4c82f6e205e7483ae20df1d1d83d9362
Aug 03 16:05:21.558248 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:21.558213    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7" event={"ID":"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e","Type":"ContainerStarted","Data":"b20a77dac495afe2fe31d4b57a249b2f4c82f6e205e7483ae20df1d1d83d9362"}
Aug 03 16:05:23.566011 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.565977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7" event={"ID":"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e","Type":"ContainerStarted","Data":"83e81f918edb3242b74e8066b2bf5b4b7eb1f1bdb1884309ba5c30cc58b69f03"}
Aug 03 16:05:23.566387 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.566028    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7" event={"ID":"daca8c9e-b40e-4c48-98ce-d3c3dc6d692e","Type":"ContainerStarted","Data":"1dec2d5a0e6fe633c3e655ac8e28c70f68adbf2bdd37e7385027e7f5af0c61ed"}
Aug 03 16:05:23.585652 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.585602    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-65jb7" podStartSLOduration=2.404268477 podStartE2EDuration="4.585587011s" totalImagesPullingTime="2.181318534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:23.584511773 +0000 UTC m=+65.851327780" watchObservedRunningTime="2026-08-03 16:05:23.585587011 +0000 UTC m=+65.852403018"
Aug 03 16:05:23.907718 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.907635    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:23.910447 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.910429    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 03 16:05:23.921073 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.921057    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 03 16:05:23.931541 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.931518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2vhk\" (UniqueName: \"kubernetes.io/projected/d377dce8-488f-49f1-9e16-9f661e0aba4a-kube-api-access-l2vhk\") pod \"network-check-target-drsdv\" (UID: \"d377dce8-488f-49f1-9e16-9f661e0aba4a\") " pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:23.967557 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.967534    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzpf\""
Aug 03 16:05:23.976077 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:23.976060    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:24.008883 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.008852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:24.009028 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.008897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:24.011949 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.011925    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 03 16:05:24.012081 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.011991    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 03 16:05:24.022120 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.021882    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab2f368-65f9-45e5-912b-c5d207c9ea22-metrics-certs\") pod \"network-metrics-daemon-sb5nd\" (UID: \"9ab2f368-65f9-45e5-912b-c5d207c9ea22\") " pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:24.025036 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.024272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c18f9f7f-3374-4214-b9e8-a98a23f5cc98-original-pull-secret\") pod \"global-pull-secret-syncer-5nw44\" (UID: \"c18f9f7f-3374-4214-b9e8-a98a23f5cc98\") " pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:24.096489 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.096458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-drsdv"]
Aug 03 16:05:24.099154 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:24.099128    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd377dce8_488f_49f1_9e16_9f661e0aba4a.slice/crio-abbc0afce3a11344466fe639943e2733fa39e2a455fbf877a79e19ced21f1d6f WatchSource:0}: Error finding container abbc0afce3a11344466fe639943e2733fa39e2a455fbf877a79e19ced21f1d6f: Status 404 returned error can't find the container with id abbc0afce3a11344466fe639943e2733fa39e2a455fbf877a79e19ced21f1d6f
Aug 03 16:05:24.254203 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.254127    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xrdkc\""
Aug 03 16:05:24.259298 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.259279    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nw44"
Aug 03 16:05:24.262413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.262397    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sb5nd"
Aug 03 16:05:24.392306 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.392276    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sb5nd"]
Aug 03 16:05:24.394892 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:24.394863    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab2f368_65f9_45e5_912b_c5d207c9ea22.slice/crio-f6f9649537a1557c12f592f20e72c8ae676de2cd05d8d8ede158839b4a56fc48 WatchSource:0}: Error finding container f6f9649537a1557c12f592f20e72c8ae676de2cd05d8d8ede158839b4a56fc48: Status 404 returned error can't find the container with id f6f9649537a1557c12f592f20e72c8ae676de2cd05d8d8ede158839b4a56fc48
Aug 03 16:05:24.412334 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.412312    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5nw44"]
Aug 03 16:05:24.414870 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:24.414844    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18f9f7f_3374_4214_b9e8_a98a23f5cc98.slice/crio-e53e2fdeceb16e33a3baf2a89067f5dce1e062be19a434ddfef80b010c7c7175 WatchSource:0}: Error finding container e53e2fdeceb16e33a3baf2a89067f5dce1e062be19a434ddfef80b010c7c7175: Status 404 returned error can't find the container with id e53e2fdeceb16e33a3baf2a89067f5dce1e062be19a434ddfef80b010c7c7175
Aug 03 16:05:24.569885 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.569848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5nw44" event={"ID":"c18f9f7f-3374-4214-b9e8-a98a23f5cc98","Type":"ContainerStarted","Data":"e53e2fdeceb16e33a3baf2a89067f5dce1e062be19a434ddfef80b010c7c7175"}
Aug 03 16:05:24.571076 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.571051    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sb5nd" event={"ID":"9ab2f368-65f9-45e5-912b-c5d207c9ea22","Type":"ContainerStarted","Data":"f6f9649537a1557c12f592f20e72c8ae676de2cd05d8d8ede158839b4a56fc48"}
Aug 03 16:05:24.572137 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:24.572119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-drsdv" event={"ID":"d377dce8-488f-49f1-9e16-9f661e0aba4a","Type":"ContainerStarted","Data":"abbc0afce3a11344466fe639943e2733fa39e2a455fbf877a79e19ced21f1d6f"}
Aug 03 16:05:25.241228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.241190    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-98mh7"]
Aug 03 16:05:25.244912 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.244884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.247705 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.247683    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 03 16:05:25.247972 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.247953    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 03 16:05:25.248265 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.248245    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 03 16:05:25.248480 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.248464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8f8d9\""
Aug 03 16:05:25.320438 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhr2p\" (UniqueName: \"kubernetes.io/projected/923f8f36-9a36-4b9d-a8c9-32c7787e7443-kube-api-access-hhr2p\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320633 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320446    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-wtmp\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320633 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320633 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-root\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320633 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923f8f36-9a36-4b9d-a8c9-32c7787e7443-metrics-client-ca\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320633 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320591    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-accelerators-collector-config\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320633 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-sys\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320971 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-tls\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.320971 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.320675    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-textfile\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.421952 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.421909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhr2p\" (UniqueName: \"kubernetes.io/projected/923f8f36-9a36-4b9d-a8c9-32c7787e7443-kube-api-access-hhr2p\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.421952 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.421954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-wtmp\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422198 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.421988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422198 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-root\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422198 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923f8f36-9a36-4b9d-a8c9-32c7787e7443-metrics-client-ca\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422198 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-accelerators-collector-config\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422198 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-sys\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422198 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422143    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-wtmp\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422198 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-tls\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422492 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-textfile\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422664 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-textfile\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422756 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422667    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-root\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422756 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923f8f36-9a36-4b9d-a8c9-32c7787e7443-sys\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.422959 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.422914    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923f8f36-9a36-4b9d-a8c9-32c7787e7443-metrics-client-ca\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.423382 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.423356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-accelerators-collector-config\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.428444 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.428421    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-tls\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.431454 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.431407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhr2p\" (UniqueName: \"kubernetes.io/projected/923f8f36-9a36-4b9d-a8c9-32c7787e7443-kube-api-access-hhr2p\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.434298 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.434269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923f8f36-9a36-4b9d-a8c9-32c7787e7443-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-98mh7\" (UID: \"923f8f36-9a36-4b9d-a8c9-32c7787e7443\") " pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.559652 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:25.559620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-98mh7"
Aug 03 16:05:25.584912 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:25.584879    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923f8f36_9a36_4b9d_a8c9_32c7787e7443.slice/crio-ae683cf4c2023e172111d453312ea65eec20510af1bb6e24fcf9c06c4188844f WatchSource:0}: Error finding container ae683cf4c2023e172111d453312ea65eec20510af1bb6e24fcf9c06c4188844f: Status 404 returned error can't find the container with id ae683cf4c2023e172111d453312ea65eec20510af1bb6e24fcf9c06c4188844f
Aug 03 16:05:26.435469 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.435064    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55b99c46b4-8qbfj"]
Aug 03 16:05:26.440621 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.439641    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.443984 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.443956    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5c8lp\""
Aug 03 16:05:26.444357 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.444177    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 03 16:05:26.444357 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.444268    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 03 16:05:26.444357 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.444318    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 03 16:05:26.444585 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.444503    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 03 16:05:26.445395 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.444700    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 03 16:05:26.445395 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.444792    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 03 16:05:26.445395 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.445298    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 03 16:05:26.451100 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.451046    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b99c46b4-8qbfj"]
Aug 03 16:05:26.452010 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.451988    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Aug 03 16:05:26.531692 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.531657    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-service-ca\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.531692 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.531692    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-console-config\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.531938 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.531718    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-oauth-serving-cert\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.531938 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.531808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67rgm\" (UniqueName: \"kubernetes.io/projected/135df6eb-3064-454d-8a39-1a1824ec28d8-kube-api-access-67rgm\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.531938 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.531862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-trusted-ca-bundle\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.531938 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.531898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-oauth-config\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.532114 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.531944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-serving-cert\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.584452 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.584411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98mh7" event={"ID":"923f8f36-9a36-4b9d-a8c9-32c7787e7443","Type":"ContainerStarted","Data":"ae683cf4c2023e172111d453312ea65eec20510af1bb6e24fcf9c06c4188844f"}
Aug 03 16:05:26.586865 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.586830    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sb5nd" event={"ID":"9ab2f368-65f9-45e5-912b-c5d207c9ea22","Type":"ContainerStarted","Data":"ebbf0357a0a028a5c3a61976f472d59d7454120f23bb9ab4899fda74d3d3a30e"}
Aug 03 16:05:26.587292 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.586890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sb5nd" event={"ID":"9ab2f368-65f9-45e5-912b-c5d207c9ea22","Type":"ContainerStarted","Data":"a3eaef800c03f430b563d953a4fd8cf3100c2362a0222f51bdd748d488593b95"}
Aug 03 16:05:26.605533 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.605479    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sb5nd" podStartSLOduration=67.420241742 podStartE2EDuration="1m8.605452842s" totalImagesPullingTime="1.1852111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:26.603949238 +0000 UTC m=+68.870765251" watchObservedRunningTime="2026-08-03 16:05:26.605452842 +0000 UTC m=+68.872268853"
Aug 03 16:05:26.632540 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.632371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-service-ca\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.632540 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.632418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-console-config\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.632775 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.632751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-oauth-serving-cert\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.632848 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.632806    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67rgm\" (UniqueName: \"kubernetes.io/projected/135df6eb-3064-454d-8a39-1a1824ec28d8-kube-api-access-67rgm\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.632848 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.632839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-trusted-ca-bundle\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.632948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.632869    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-oauth-config\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.632948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.632913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-serving-cert\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.633831 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.633706    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-service-ca\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.633831 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.633711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-console-config\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.634283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.634252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-oauth-serving-cert\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.634580 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.634558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-trusted-ca-bundle\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.636436 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.636417    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-serving-cert\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.637092 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.637067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-oauth-config\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.641940 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.641909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67rgm\" (UniqueName: \"kubernetes.io/projected/135df6eb-3064-454d-8a39-1a1824ec28d8-kube-api-access-67rgm\") pod \"console-55b99c46b4-8qbfj\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") " pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.757422 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.757386    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:26.905315 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:26.905293    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b99c46b4-8qbfj"]
Aug 03 16:05:27.548356 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:27.548326    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5zd6c"
Aug 03 16:05:27.593971 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:27.593938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98mh7" event={"ID":"923f8f36-9a36-4b9d-a8c9-32c7787e7443","Type":"ContainerStarted","Data":"cfc0f9cb84eeb3ea5ef08c2e8aa323baa00f969b8877a978a043679d15ec9d9a"}
Aug 03 16:05:28.598588 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:28.598553    2564 generic.go:356] "Generic (PLEG): container finished" podID="923f8f36-9a36-4b9d-a8c9-32c7787e7443" containerID="cfc0f9cb84eeb3ea5ef08c2e8aa323baa00f969b8877a978a043679d15ec9d9a" exitCode=0
Aug 03 16:05:28.599053 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:28.598651    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98mh7" event={"ID":"923f8f36-9a36-4b9d-a8c9-32c7787e7443","Type":"ContainerDied","Data":"cfc0f9cb84eeb3ea5ef08c2e8aa323baa00f969b8877a978a043679d15ec9d9a"}
Aug 03 16:05:29.293474 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:29.293428    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod135df6eb_3064_454d_8a39_1a1824ec28d8.slice/crio-aa95b8df57af2782f833f69464a197c2f7967606cb50ce3ed0609e409d31b45a WatchSource:0}: Error finding container aa95b8df57af2782f833f69464a197c2f7967606cb50ce3ed0609e409d31b45a: Status 404 returned error can't find the container with id aa95b8df57af2782f833f69464a197c2f7967606cb50ce3ed0609e409d31b45a
Aug 03 16:05:29.603371 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.603338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b99c46b4-8qbfj" event={"ID":"135df6eb-3064-454d-8a39-1a1824ec28d8","Type":"ContainerStarted","Data":"aa95b8df57af2782f833f69464a197c2f7967606cb50ce3ed0609e409d31b45a"}
Aug 03 16:05:29.635242 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.635211    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-785f6df764-q8vqj"]
Aug 03 16:05:29.656906 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.656887    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-785f6df764-q8vqj"]
Aug 03 16:05:29.657013 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.656980    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.659689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.659669    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Aug 03 16:05:29.661090 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.661067    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Aug 03 16:05:29.661090 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.661087    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tdhfs\""
Aug 03 16:05:29.661283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.661126    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f1knifmto9d68\""
Aug 03 16:05:29.661460 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.661343    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Aug 03 16:05:29.661460 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.661424    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 03 16:05:29.759863 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.759822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8fg7\" (UniqueName: \"kubernetes.io/projected/f4e83f09-5e58-4e5a-8416-262e1f876f2f-kube-api-access-w8fg7\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.759966 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.759875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-secret-metrics-server-tls\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.759966 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.759908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f4e83f09-5e58-4e5a-8416-262e1f876f2f-metrics-server-audit-profiles\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.759966 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.759939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f4e83f09-5e58-4e5a-8416-262e1f876f2f-audit-log\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.760299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.759975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e83f09-5e58-4e5a-8416-262e1f876f2f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.760299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.760077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-client-ca-bundle\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.760299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.760112    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-secret-metrics-server-client-certs\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861047 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861013    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8fg7\" (UniqueName: \"kubernetes.io/projected/f4e83f09-5e58-4e5a-8416-262e1f876f2f-kube-api-access-w8fg7\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861154 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-secret-metrics-server-tls\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861154 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f4e83f09-5e58-4e5a-8416-262e1f876f2f-metrics-server-audit-profiles\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861154 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f4e83f09-5e58-4e5a-8416-262e1f876f2f-audit-log\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861154 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e83f09-5e58-4e5a-8416-262e1f876f2f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861358 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-client-ca-bundle\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861358 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-secret-metrics-server-client-certs\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.861627 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.861608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f4e83f09-5e58-4e5a-8416-262e1f876f2f-audit-log\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.862060 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.862032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e83f09-5e58-4e5a-8416-262e1f876f2f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.863522 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.862274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f4e83f09-5e58-4e5a-8416-262e1f876f2f-metrics-server-audit-profiles\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.863892 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.863871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-secret-metrics-server-tls\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.863990 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.863942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-secret-metrics-server-client-certs\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.864215 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.864195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e83f09-5e58-4e5a-8416-262e1f876f2f-client-ca-bundle\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.870688 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.870670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8fg7\" (UniqueName: \"kubernetes.io/projected/f4e83f09-5e58-4e5a-8416-262e1f876f2f-kube-api-access-w8fg7\") pod \"metrics-server-785f6df764-q8vqj\" (UID: \"f4e83f09-5e58-4e5a-8416-262e1f876f2f\") " pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.969318 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.969279    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:29.972912 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.972883    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"]
Aug 03 16:05:29.997813 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.997785    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"]
Aug 03 16:05:29.997942 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:29.997905    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.000488 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.000428    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Aug 03 16:05:30.000488 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.000462    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-r226f\""
Aug 03 16:05:30.062923 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.062890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drjfs\" (UniqueName: \"kubernetes.io/projected/4381cea8-6c83-482c-8809-34e69fc0f355-kube-api-access-drjfs\") pod \"monitoring-plugin-6dffbbd67-zkvmx\" (UID: \"4381cea8-6c83-482c-8809-34e69fc0f355\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.063045 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.062928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4381cea8-6c83-482c-8809-34e69fc0f355-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-zkvmx\" (UID: \"4381cea8-6c83-482c-8809-34e69fc0f355\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.118885 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.118791    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-785f6df764-q8vqj"]
Aug 03 16:05:30.121336 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:30.121310    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e83f09_5e58_4e5a_8416_262e1f876f2f.slice/crio-4557fd05081733bc11ea4f5218873ff72c8ea27055f93e32cfc23ea08216da72 WatchSource:0}: Error finding container 4557fd05081733bc11ea4f5218873ff72c8ea27055f93e32cfc23ea08216da72: Status 404 returned error can't find the container with id 4557fd05081733bc11ea4f5218873ff72c8ea27055f93e32cfc23ea08216da72
Aug 03 16:05:30.164156 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.164131    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drjfs\" (UniqueName: \"kubernetes.io/projected/4381cea8-6c83-482c-8809-34e69fc0f355-kube-api-access-drjfs\") pod \"monitoring-plugin-6dffbbd67-zkvmx\" (UID: \"4381cea8-6c83-482c-8809-34e69fc0f355\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.164249 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.164164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4381cea8-6c83-482c-8809-34e69fc0f355-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-zkvmx\" (UID: \"4381cea8-6c83-482c-8809-34e69fc0f355\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.164302 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:30.164284    2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Aug 03 16:05:30.164347 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:05:30.164337    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4381cea8-6c83-482c-8809-34e69fc0f355-monitoring-plugin-cert podName:4381cea8-6c83-482c-8809-34e69fc0f355 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:30.664318725 +0000 UTC m=+72.931134711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/4381cea8-6c83-482c-8809-34e69fc0f355-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-zkvmx" (UID: "4381cea8-6c83-482c-8809-34e69fc0f355") : secret "monitoring-plugin-cert" not found
Aug 03 16:05:30.175295 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.175270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drjfs\" (UniqueName: \"kubernetes.io/projected/4381cea8-6c83-482c-8809-34e69fc0f355-kube-api-access-drjfs\") pod \"monitoring-plugin-6dffbbd67-zkvmx\" (UID: \"4381cea8-6c83-482c-8809-34e69fc0f355\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.609142 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.609113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98mh7" event={"ID":"923f8f36-9a36-4b9d-a8c9-32c7787e7443","Type":"ContainerStarted","Data":"d4b6f8e8ff0b1536a5459f8726a480f3b9496605b31bf9ac42081dbc463bd797"}
Aug 03 16:05:30.609660 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.609171    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98mh7" event={"ID":"923f8f36-9a36-4b9d-a8c9-32c7787e7443","Type":"ContainerStarted","Data":"568456f173630daf36fb5ba7055d1625e4da63f217e64ed1cd5c61c7fc21a7cd"}
Aug 03 16:05:30.610450 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.610421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5nw44" event={"ID":"c18f9f7f-3374-4214-b9e8-a98a23f5cc98","Type":"ContainerStarted","Data":"26228b9ecfe06c4de16da91caba94ac98cc4358650aebefa255de7f335236f71"}
Aug 03 16:05:30.611452 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.611427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj" event={"ID":"f4e83f09-5e58-4e5a-8416-262e1f876f2f","Type":"ContainerStarted","Data":"4557fd05081733bc11ea4f5218873ff72c8ea27055f93e32cfc23ea08216da72"}
Aug 03 16:05:30.612645 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.612626    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-drsdv" event={"ID":"d377dce8-488f-49f1-9e16-9f661e0aba4a","Type":"ContainerStarted","Data":"231891e2e7a81974a1489ae61eef640cb43086a40b7be4350c257d6951c3d1bf"}
Aug 03 16:05:30.612765 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.612754    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:05:30.635248 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.635206    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-98mh7" podStartSLOduration=4.530657179 podStartE2EDuration="5.635194182s" totalImagesPullingTime="1.023413671s" totalInitContainerRuntime="81.123332ms" isStatefulPod=true podCreationTimestamp="2026-08-03 16:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:30.633677392 +0000 UTC m=+72.900493400" watchObservedRunningTime="2026-08-03 16:05:30.635194182 +0000 UTC m=+72.902010189"
Aug 03 16:05:30.656408 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.656370    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-drsdv" podStartSLOduration=68.106241863 podStartE2EDuration="1m13.656357681s" totalImagesPullingTime="5.550115818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:30.65539977 +0000 UTC m=+72.922215777" watchObservedRunningTime="2026-08-03 16:05:30.656357681 +0000 UTC m=+72.923173723"
Aug 03 16:05:30.668836 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.668807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4381cea8-6c83-482c-8809-34e69fc0f355-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-zkvmx\" (UID: \"4381cea8-6c83-482c-8809-34e69fc0f355\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.671237 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.671218    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4381cea8-6c83-482c-8809-34e69fc0f355-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-zkvmx\" (UID: \"4381cea8-6c83-482c-8809-34e69fc0f355\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:30.674657 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.674617    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5nw44" podStartSLOduration=67.425956039 podStartE2EDuration="1m12.674604764s" totalImagesPullingTime="5.248648725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:30.673902737 +0000 UTC m=+72.940718745" watchObservedRunningTime="2026-08-03 16:05:30.674604764 +0000 UTC m=+72.941420748"
Aug 03 16:05:30.910997 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:30.910920    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:31.039651 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.039612    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"]
Aug 03 16:05:31.043701 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:31.043674    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4381cea8_6c83_482c_8809_34e69fc0f355.slice/crio-8569941902c0ef1c99e1f4415802aa9f90f0cee9878e22ae1752e03fb75abb09 WatchSource:0}: Error finding container 8569941902c0ef1c99e1f4415802aa9f90f0cee9878e22ae1752e03fb75abb09: Status 404 returned error can't find the container with id 8569941902c0ef1c99e1f4415802aa9f90f0cee9878e22ae1752e03fb75abb09
Aug 03 16:05:31.425104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.423687    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:05:31.427968 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.427936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.432520 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.432219    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 03 16:05:31.432520 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.432336    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 03 16:05:31.432520 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.432342    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 03 16:05:31.432520 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.432343    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-c6bkb\""
Aug 03 16:05:31.432520 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.432440    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 03 16:05:31.432817 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.432660    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 03 16:05:31.433332 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.433311    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 03 16:05:31.433574 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.433548    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 03 16:05:31.433727 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.433714    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 03 16:05:31.434319 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.434151    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 03 16:05:31.434319 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.434208    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 03 16:05:31.434573 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.434554    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 03 16:05:31.434649 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.434621    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-36gtabm7dh64k\""
Aug 03 16:05:31.434813 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.434793    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 03 16:05:31.434916 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.434898    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 03 16:05:31.436150 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.436112    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 03 16:05:31.445215 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.445189    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:05:31.575932 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.575890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576114 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.575943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576114 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.575983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-web-config\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576114 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576114 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576114 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnrdl\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-kube-api-access-jnrdl\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576146    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576194    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576377    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-config\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576411    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576818 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576818 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-config-out\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.576818 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.576523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.618656 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.618587    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx" event={"ID":"4381cea8-6c83-482c-8809-34e69fc0f355","Type":"ContainerStarted","Data":"8569941902c0ef1c99e1f4415802aa9f90f0cee9878e22ae1752e03fb75abb09"}
Aug 03 16:05:31.677864 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.677864 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.677864 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-web-config\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnrdl\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-kube-api-access-jnrdl\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.677982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-config\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-config-out\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.678938 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.678652    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.681144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.679107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.681144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.679673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.681144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.680631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.681383 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.681272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.683914 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.683889    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-config-out\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.684398 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.684371    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.685143 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.685116    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.685236 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.685204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.687876 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.687810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.687876 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.687814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-web-config\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.688303 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.688262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.688394 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.688344    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.688644 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.688609    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.689232 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.689206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.689898 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.689846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.689898 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.689870    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-config\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.690312 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.690275    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.691837 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.691810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.692393 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.692363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnrdl\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-kube-api-access-jnrdl\") pod \"prometheus-k8s-0\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.746052 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.746015    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:31.959623 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:31.959472    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:05:31.965300 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:05:31.965275    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df08167_f90a_4016_9a7a_56ad8b95c92b.slice/crio-7205fb57d8590127b59f95843e6f92b1efe1f4a51aa480284e8394633ee193cb WatchSource:0}: Error finding container 7205fb57d8590127b59f95843e6f92b1efe1f4a51aa480284e8394633ee193cb: Status 404 returned error can't find the container with id 7205fb57d8590127b59f95843e6f92b1efe1f4a51aa480284e8394633ee193cb
Aug 03 16:05:32.249139 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:32.249055    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b99c46b4-8qbfj"]
Aug 03 16:05:32.626937 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:32.626823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerStarted","Data":"7205fb57d8590127b59f95843e6f92b1efe1f4a51aa480284e8394633ee193cb"}
Aug 03 16:05:32.631115 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:32.630495    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj" event={"ID":"f4e83f09-5e58-4e5a-8416-262e1f876f2f","Type":"ContainerStarted","Data":"025c6a87021e7fc3c0508566d01395e072482dd2b9bc8e7ed81be41c2c4189fe"}
Aug 03 16:05:32.650806 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:32.650723    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj" podStartSLOduration=1.992749809 podStartE2EDuration="3.650703706s" totalImagesPullingTime="1.657953897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:32.649130361 +0000 UTC m=+74.915946370" watchObservedRunningTime="2026-08-03 16:05:32.650703706 +0000 UTC m=+74.917519715"
Aug 03 16:05:33.636224 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:33.636190    2564 generic.go:356] "Generic (PLEG): container finished" podID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerID="cfbe63d7a763fb00a56596dbe68e0870804de1e7bffcb851f5a86105bc3a9ace" exitCode=0
Aug 03 16:05:33.636685 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:33.636283    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"cfbe63d7a763fb00a56596dbe68e0870804de1e7bffcb851f5a86105bc3a9ace"}
Aug 03 16:05:33.638080 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:33.638047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx" event={"ID":"4381cea8-6c83-482c-8809-34e69fc0f355","Type":"ContainerStarted","Data":"c7664e2b94d51a9aad793908016c6b1b4820557f1a7ca0e26796d92d1e8fd703"}
Aug 03 16:05:33.681680 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:33.681511    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx" podStartSLOduration=2.582291839 podStartE2EDuration="4.681493065s" totalImagesPullingTime="2.099201226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:33.681187117 +0000 UTC m=+75.948003125" watchObservedRunningTime="2026-08-03 16:05:33.681493065 +0000 UTC m=+75.948309074"
Aug 03 16:05:33.883164 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:33.883115    2564 patch_prober.go:28] interesting pod/image-registry-58568488cc-mxl9t container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Aug 03 16:05:33.883335 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:33.883198    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-58568488cc-mxl9t" podUID="f52dae94-350b-4c29-86a5-c8cd5440f5c7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 03 16:05:34.644999 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:34.644890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b99c46b4-8qbfj" event={"ID":"135df6eb-3064-454d-8a39-1a1824ec28d8","Type":"ContainerStarted","Data":"bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790"}
Aug 03 16:05:34.645438 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:34.645173    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:34.651214 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:34.651188    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-zkvmx"
Aug 03 16:05:34.664314 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:34.664264    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55b99c46b4-8qbfj" podStartSLOduration=3.8856204930000002 podStartE2EDuration="8.664246913s" totalImagesPullingTime="4.77862642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:34.664143995 +0000 UTC m=+76.930960002" watchObservedRunningTime="2026-08-03 16:05:34.664246913 +0000 UTC m=+76.931062924"
Aug 03 16:05:35.534180 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:35.534150    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58568488cc-mxl9t"
Aug 03 16:05:36.757560 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:36.757520    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:05:37.659664 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:37.659632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerStarted","Data":"05dac5e0c23438b9e93a3b2a521b709134f31ebe038cf479c69ff906d0de1486"}
Aug 03 16:05:37.659823 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:37.659694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerStarted","Data":"7a34442e174a74116e2a7dcf88ef73627f87de703ba2e3ed2ee8d875dd1c6eb3"}
Aug 03 16:05:39.670843 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:39.670804    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerStarted","Data":"ecb657bd85e5b366e4211d2b2e2cce8b1addbdcaa810978f796e5dbc3185c00f"}
Aug 03 16:05:39.671235 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:39.670875    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerStarted","Data":"49d6214e210b266efe455a310abca4539a66703241d760beacb6b6d852d7ef58"}
Aug 03 16:05:39.671235 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:39.670904    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerStarted","Data":"d570c23c7c4bccab11ba9fc65b14545238438c2644f48ed43a51cb4e2782c12e"}
Aug 03 16:05:39.671235 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:39.670931    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerStarted","Data":"7087258375fbb485d3ee786deabdcfd6e9ba84f96b719a1fcd4503bab91000e7"}
Aug 03 16:05:39.701126 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:39.701016    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.087714957 podStartE2EDuration="8.700993733s" totalImagesPullingTime="6.58899522s" totalInitContainerRuntime="24.283556ms" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:39.698938819 +0000 UTC m=+81.965754823" watchObservedRunningTime="2026-08-03 16:05:39.700993733 +0000 UTC m=+81.967809741"
Aug 03 16:05:41.747104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:41.747068    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:05:49.970039 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:49.970005    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:49.970428 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:49.970065    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:05:59.666754 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:59.666691    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55b99c46b4-8qbfj" podUID="135df6eb-3064-454d-8a39-1a1824ec28d8" containerName="console" containerID="cri-o://bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790" gracePeriod=15
Aug 03 16:05:59.945718 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:05:59.945656    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:06:00.013309 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.013281    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-console-config\") pod \"135df6eb-3064-454d-8a39-1a1824ec28d8\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") "
Aug 03 16:06:00.013309 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.013317    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-service-ca\") pod \"135df6eb-3064-454d-8a39-1a1824ec28d8\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") "
Aug 03 16:06:00.013546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.013336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-oauth-serving-cert\") pod \"135df6eb-3064-454d-8a39-1a1824ec28d8\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") "
Aug 03 16:06:00.013546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.013371    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-oauth-config\") pod \"135df6eb-3064-454d-8a39-1a1824ec28d8\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") "
Aug 03 16:06:00.013546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.013482    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-serving-cert\") pod \"135df6eb-3064-454d-8a39-1a1824ec28d8\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") "
Aug 03 16:06:00.013689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.013546    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-trusted-ca-bundle\") pod \"135df6eb-3064-454d-8a39-1a1824ec28d8\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") "
Aug 03 16:06:00.013689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.013606    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67rgm\" (UniqueName: \"kubernetes.io/projected/135df6eb-3064-454d-8a39-1a1824ec28d8-kube-api-access-67rgm\") pod \"135df6eb-3064-454d-8a39-1a1824ec28d8\" (UID: \"135df6eb-3064-454d-8a39-1a1824ec28d8\") "
Aug 03 16:06:00.014070 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.014015    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "135df6eb-3064-454d-8a39-1a1824ec28d8" (UID: "135df6eb-3064-454d-8a39-1a1824ec28d8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:00.014070 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.014026    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-service-ca" (OuterVolumeSpecName: "service-ca") pod "135df6eb-3064-454d-8a39-1a1824ec28d8" (UID: "135df6eb-3064-454d-8a39-1a1824ec28d8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:00.014070 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.014034    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-console-config" (OuterVolumeSpecName: "console-config") pod "135df6eb-3064-454d-8a39-1a1824ec28d8" (UID: "135df6eb-3064-454d-8a39-1a1824ec28d8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:00.014070 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.014044    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "135df6eb-3064-454d-8a39-1a1824ec28d8" (UID: "135df6eb-3064-454d-8a39-1a1824ec28d8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:00.015848 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.015809    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "135df6eb-3064-454d-8a39-1a1824ec28d8" (UID: "135df6eb-3064-454d-8a39-1a1824ec28d8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:00.015954 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.015870    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/135df6eb-3064-454d-8a39-1a1824ec28d8-kube-api-access-67rgm" (OuterVolumeSpecName: "kube-api-access-67rgm") pod "135df6eb-3064-454d-8a39-1a1824ec28d8" (UID: "135df6eb-3064-454d-8a39-1a1824ec28d8"). InnerVolumeSpecName "kube-api-access-67rgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:06:00.015954 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.015877    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "135df6eb-3064-454d-8a39-1a1824ec28d8" (UID: "135df6eb-3064-454d-8a39-1a1824ec28d8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:00.114970 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.114939    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67rgm\" (UniqueName: \"kubernetes.io/projected/135df6eb-3064-454d-8a39-1a1824ec28d8-kube-api-access-67rgm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:00.114970 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.114963    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-console-config\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:00.114970 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.114973    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-service-ca\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:00.115175 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.114981    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-oauth-serving-cert\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:00.115175 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.114990    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-oauth-config\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:00.115175 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.114998    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/135df6eb-3064-454d-8a39-1a1824ec28d8-console-serving-cert\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:00.115175 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.115006    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/135df6eb-3064-454d-8a39-1a1824ec28d8-trusted-ca-bundle\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:00.748970 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.748938    2564 generic.go:356] "Generic (PLEG): container finished" podID="135df6eb-3064-454d-8a39-1a1824ec28d8" containerID="bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790" exitCode=0
Aug 03 16:06:00.749421 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.748985    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b99c46b4-8qbfj"
Aug 03 16:06:00.749421 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.749027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b99c46b4-8qbfj" event={"ID":"135df6eb-3064-454d-8a39-1a1824ec28d8","Type":"ContainerDied","Data":"bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790"}
Aug 03 16:06:00.749421 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.749080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b99c46b4-8qbfj" event={"ID":"135df6eb-3064-454d-8a39-1a1824ec28d8","Type":"ContainerDied","Data":"aa95b8df57af2782f833f69464a197c2f7967606cb50ce3ed0609e409d31b45a"}
Aug 03 16:06:00.749421 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.749101    2564 scope.go:117] "RemoveContainer" containerID="bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790"
Aug 03 16:06:00.757135 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.757120    2564 scope.go:117] "RemoveContainer" containerID="bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790"
Aug 03 16:06:00.757374 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:06:00.757355    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790\": container with ID starting with bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790 not found: ID does not exist" containerID="bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790"
Aug 03 16:06:00.757422 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.757384    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790"} err="failed to get container status \"bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790\": rpc error: code = NotFound desc = could not find container \"bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790\": container with ID starting with bcff6056347b283cd3dba4311ef1fea1b35d45a3f2d24601d444ce6e41b15790 not found: ID does not exist"
Aug 03 16:06:00.768663 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.768641    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b99c46b4-8qbfj"]
Aug 03 16:06:00.772846 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:00.772822    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55b99c46b4-8qbfj"]
Aug 03 16:06:01.621477 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:01.621450    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-drsdv"
Aug 03 16:06:02.246476 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:02.246438    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="135df6eb-3064-454d-8a39-1a1824ec28d8" path="/var/lib/kubelet/pods/135df6eb-3064-454d-8a39-1a1824ec28d8/volumes"
Aug 03 16:06:09.978839 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:09.978810    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:06:09.982653 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:09.982628    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-785f6df764-q8vqj"
Aug 03 16:06:31.746896 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:31.746859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:31.765575 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:31.765545    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:31.872517 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:31.872491    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:49.771785 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.771727    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:06:49.772329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.772052    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="prometheus" containerID="cri-o://7a34442e174a74116e2a7dcf88ef73627f87de703ba2e3ed2ee8d875dd1c6eb3" gracePeriod=600
Aug 03 16:06:49.772329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.772104    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy" containerID="cri-o://49d6214e210b266efe455a310abca4539a66703241d760beacb6b6d852d7ef58" gracePeriod=600
Aug 03 16:06:49.772329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.772129    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ecb657bd85e5b366e4211d2b2e2cce8b1addbdcaa810978f796e5dbc3185c00f" gracePeriod=600
Aug 03 16:06:49.772329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.772158    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="config-reloader" containerID="cri-o://05dac5e0c23438b9e93a3b2a521b709134f31ebe038cf479c69ff906d0de1486" gracePeriod=600
Aug 03 16:06:49.772329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.772122    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="thanos-sidecar" containerID="cri-o://7087258375fbb485d3ee786deabdcfd6e9ba84f96b719a1fcd4503bab91000e7" gracePeriod=600
Aug 03 16:06:49.772329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.772147    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-web" containerID="cri-o://d570c23c7c4bccab11ba9fc65b14545238438c2644f48ed43a51cb4e2782c12e" gracePeriod=600
Aug 03 16:06:49.925333 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925307    2564 generic.go:356] "Generic (PLEG): container finished" podID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerID="ecb657bd85e5b366e4211d2b2e2cce8b1addbdcaa810978f796e5dbc3185c00f" exitCode=0
Aug 03 16:06:49.925333 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925328    2564 generic.go:356] "Generic (PLEG): container finished" podID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerID="49d6214e210b266efe455a310abca4539a66703241d760beacb6b6d852d7ef58" exitCode=0
Aug 03 16:06:49.925333 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925335    2564 generic.go:356] "Generic (PLEG): container finished" podID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerID="7087258375fbb485d3ee786deabdcfd6e9ba84f96b719a1fcd4503bab91000e7" exitCode=0
Aug 03 16:06:49.925521 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925347    2564 generic.go:356] "Generic (PLEG): container finished" podID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerID="05dac5e0c23438b9e93a3b2a521b709134f31ebe038cf479c69ff906d0de1486" exitCode=0
Aug 03 16:06:49.925521 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925353    2564 generic.go:356] "Generic (PLEG): container finished" podID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerID="7a34442e174a74116e2a7dcf88ef73627f87de703ba2e3ed2ee8d875dd1c6eb3" exitCode=0
Aug 03 16:06:49.925521 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"ecb657bd85e5b366e4211d2b2e2cce8b1addbdcaa810978f796e5dbc3185c00f"}
Aug 03 16:06:49.925521 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"49d6214e210b266efe455a310abca4539a66703241d760beacb6b6d852d7ef58"}
Aug 03 16:06:49.925521 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"7087258375fbb485d3ee786deabdcfd6e9ba84f96b719a1fcd4503bab91000e7"}
Aug 03 16:06:49.925521 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"05dac5e0c23438b9e93a3b2a521b709134f31ebe038cf479c69ff906d0de1486"}
Aug 03 16:06:49.925689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:49.925528    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"7a34442e174a74116e2a7dcf88ef73627f87de703ba2e3ed2ee8d875dd1c6eb3"}
Aug 03 16:06:50.933799 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:50.933766    2564 generic.go:356] "Generic (PLEG): container finished" podID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerID="d570c23c7c4bccab11ba9fc65b14545238438c2644f48ed43a51cb4e2782c12e" exitCode=0
Aug 03 16:06:50.934151 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:50.933824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"d570c23c7c4bccab11ba9fc65b14545238438c2644f48ed43a51cb4e2782c12e"}
Aug 03 16:06:51.012540 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.012518    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:51.111039 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.110966    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-0\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111039 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111004    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-serving-certs-ca-bundle\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111039 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111033    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-web-config\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111054    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-db\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111108    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnrdl\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-kube-api-access-jnrdl\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111146    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-thanos-prometheus-http-client-file\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-2\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111283 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111268    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-metrics-client-certs\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111298    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-kube-rbac-proxy\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-grpc-tls\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111365    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-kubelet-serving-ca-bundle\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111394    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-config\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111412    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111424    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-metrics-client-ca\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111491    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.111587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111541    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-1\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.112020 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111680    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-tls\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.112020 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111716    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-config-out\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.112020 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111770    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-trusted-ca-bundle\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.112020 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111806    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-tls-assets\") pod \"5df08167-f90a-4016-9a7a-56ad8b95c92b\" (UID: \"5df08167-f90a-4016-9a7a-56ad8b95c92b\") "
Aug 03 16:06:51.112235 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.112048    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.114144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.111768    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:51.114144 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.112406    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:06:51.114322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.114145    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:51.114322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.112639    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:51.114322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.113221    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:51.114322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.113306    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:51.114322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.113732    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:51.114322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.114046    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.114322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.114115    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-kube-api-access-jnrdl" (OuterVolumeSpecName: "kube-api-access-jnrdl") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "kube-api-access-jnrdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:06:51.115433 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.115376    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.115878 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.115828    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.116125 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.116106    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-config-out" (OuterVolumeSpecName: "config-out") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:06:51.116211 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.116149    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.116271 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.116221    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:06:51.116271 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.116257    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-config" (OuterVolumeSpecName: "config") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.116719 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.116691    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.116811 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.116713    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.117068 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.117043    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.124409 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.124385    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-web-config" (OuterVolumeSpecName: "web-config") pod "5df08167-f90a-4016-9a7a-56ad8b95c92b" (UID: "5df08167-f90a-4016-9a7a-56ad8b95c92b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:51.212834 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212794    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.212834 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212828    2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-config\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.212834 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212840    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-configmap-metrics-client-ca\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.212834 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212849    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212859    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212868    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212876    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-config-out\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212885    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212895    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-tls-assets\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212903    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212911    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-web-config\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212920    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-db\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212928    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnrdl\" (UniqueName: \"kubernetes.io/projected/5df08167-f90a-4016-9a7a-56ad8b95c92b-kube-api-access-jnrdl\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212936    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212945    2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212963    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5df08167-f90a-4016-9a7a-56ad8b95c92b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212984    2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-metrics-client-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.212993    2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-kube-rbac-proxy\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.213083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.213000    2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5df08167-f90a-4016-9a7a-56ad8b95c92b-secret-grpc-tls\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:06:51.940904 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.940870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5df08167-f90a-4016-9a7a-56ad8b95c92b","Type":"ContainerDied","Data":"7205fb57d8590127b59f95843e6f92b1efe1f4a51aa480284e8394633ee193cb"}
Aug 03 16:06:51.941296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.940912    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:51.941296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.940956    2564 scope.go:117] "RemoveContainer" containerID="ecb657bd85e5b366e4211d2b2e2cce8b1addbdcaa810978f796e5dbc3185c00f"
Aug 03 16:06:51.950663 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.950644    2564 scope.go:117] "RemoveContainer" containerID="49d6214e210b266efe455a310abca4539a66703241d760beacb6b6d852d7ef58"
Aug 03 16:06:51.957305 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.957285    2564 scope.go:117] "RemoveContainer" containerID="d570c23c7c4bccab11ba9fc65b14545238438c2644f48ed43a51cb4e2782c12e"
Aug 03 16:06:51.963613 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.963598    2564 scope.go:117] "RemoveContainer" containerID="7087258375fbb485d3ee786deabdcfd6e9ba84f96b719a1fcd4503bab91000e7"
Aug 03 16:06:51.968240 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.968219    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:06:51.970261 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.970245    2564 scope.go:117] "RemoveContainer" containerID="05dac5e0c23438b9e93a3b2a521b709134f31ebe038cf479c69ff906d0de1486"
Aug 03 16:06:51.973158 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.973139    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:06:51.976715 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.976697    2564 scope.go:117] "RemoveContainer" containerID="7a34442e174a74116e2a7dcf88ef73627f87de703ba2e3ed2ee8d875dd1c6eb3"
Aug 03 16:06:51.983092 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.983075    2564 scope.go:117] "RemoveContainer" containerID="cfbe63d7a763fb00a56596dbe68e0870804de1e7bffcb851f5a86105bc3a9ace"
Aug 03 16:06:51.999837 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:51.999815    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:06:52.000049 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000038    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="prometheus"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000051    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="prometheus"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000059    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="135df6eb-3064-454d-8a39-1a1824ec28d8" containerName="console"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000066    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="135df6eb-3064-454d-8a39-1a1824ec28d8" containerName="console"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000072    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="config-reloader"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000078    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="config-reloader"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000084    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="thanos-sidecar"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000090    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="thanos-sidecar"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000096    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy"
Aug 03 16:06:52.000102 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000101    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000107    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="init-config-reloader"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000113    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="init-config-reloader"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000118    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-web"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000123    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-web"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000127    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-thanos"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000132    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-thanos"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000170    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="config-reloader"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000177    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="thanos-sidecar"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000183    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="prometheus"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000188    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000194    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-thanos"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000199    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="135df6eb-3064-454d-8a39-1a1824ec28d8" containerName="console"
Aug 03 16:06:52.000350 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.000205    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" containerName="kube-rbac-proxy-web"
Aug 03 16:06:52.005126 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.005111    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.008025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008005    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 03 16:06:52.008025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008021    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 03 16:06:52.008173 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008035    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 03 16:06:52.008173 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008135    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-36gtabm7dh64k\""
Aug 03 16:06:52.008268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008256    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 03 16:06:52.008322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008274    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 03 16:06:52.008322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008289    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 03 16:06:52.008322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008294    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 03 16:06:52.008497 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008480    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 03 16:06:52.008569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008483    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-c6bkb\""
Aug 03 16:06:52.009244 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.008981    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 03 16:06:52.010383 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.009878    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 03 16:06:52.010533 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.010515    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 03 16:06:52.010883 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.010860    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 03 16:06:52.016295 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.016265    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 03 16:06:52.017210 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.017195    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 03 16:06:52.018749 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018702    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.018749 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.018929 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018787    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.018929 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.018929 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.018929 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.018929 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018940    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a56f11f-47f6-440f-9071-f0fd0d910489-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.018990    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh8r7\" (UniqueName: \"kubernetes.io/projected/0a56f11f-47f6-440f-9071-f0fd0d910489-kube-api-access-bh8r7\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-config\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019148    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a56f11f-47f6-440f-9071-f0fd0d910489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.019563 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.019274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.023171 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.023150    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:06:52.120212 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120131    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120212 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120212 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120212 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120202    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a56f11f-47f6-440f-9071-f0fd0d910489-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8r7\" (UniqueName: \"kubernetes.io/projected/0a56f11f-47f6-440f-9071-f0fd0d910489-kube-api-access-bh8r7\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-config\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120519 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a56f11f-47f6-440f-9071-f0fd0d910489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.120978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.120865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.121425 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.121409    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.121491 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.121436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.122323 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.122296    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.123011 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.122984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.123119 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.123075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a56f11f-47f6-440f-9071-f0fd0d910489-config-out\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.123399 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.123374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.123877 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.123512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.124054 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.124032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-config\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.124465 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.124205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.124465 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.124400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.124465 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.124433    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.124683 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.124624    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a56f11f-47f6-440f-9071-f0fd0d910489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.125604 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.125573    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a56f11f-47f6-440f-9071-f0fd0d910489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.125695 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.125628    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.126013 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.125996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.126345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.126329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.126729 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.126715    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a56f11f-47f6-440f-9071-f0fd0d910489-web-config\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.130708 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.130685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh8r7\" (UniqueName: \"kubernetes.io/projected/0a56f11f-47f6-440f-9071-f0fd0d910489-kube-api-access-bh8r7\") pod \"prometheus-k8s-0\" (UID: \"0a56f11f-47f6-440f-9071-f0fd0d910489\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.242041 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.242011    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5df08167-f90a-4016-9a7a-56ad8b95c92b" path="/var/lib/kubelet/pods/5df08167-f90a-4016-9a7a-56ad8b95c92b/volumes"
Aug 03 16:06:52.317620 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.317597    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:06:52.455202 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.455178    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 03 16:06:52.457421 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:06:52.457394    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a56f11f_47f6_440f_9071_f0fd0d910489.slice/crio-a3a33953b774504d6613bffaa8b01c5d5fe8808b818c4a5531d490280eb607ff WatchSource:0}: Error finding container a3a33953b774504d6613bffaa8b01c5d5fe8808b818c4a5531d490280eb607ff: Status 404 returned error can't find the container with id a3a33953b774504d6613bffaa8b01c5d5fe8808b818c4a5531d490280eb607ff
Aug 03 16:06:52.945308 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.945271    2564 generic.go:356] "Generic (PLEG): container finished" podID="0a56f11f-47f6-440f-9071-f0fd0d910489" containerID="814d5378e7ee7be28a302c0ea149ca7a7ff233d919acde8e557b78d9a349f517" exitCode=0
Aug 03 16:06:52.945826 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.945365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerDied","Data":"814d5378e7ee7be28a302c0ea149ca7a7ff233d919acde8e557b78d9a349f517"}
Aug 03 16:06:52.945826 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:52.945465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerStarted","Data":"a3a33953b774504d6613bffaa8b01c5d5fe8808b818c4a5531d490280eb607ff"}
Aug 03 16:06:53.953202 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:53.953165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerStarted","Data":"3daf10013dc73e06e5617db0748d96e2b13235b8aa0433f2fb908a845dd973f8"}
Aug 03 16:06:53.953672 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:53.953243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerStarted","Data":"98a0cb3dbe147bae310845bf50cbbb7b0fc2931670c6c53fbad4dd23c29226e6"}
Aug 03 16:06:53.953672 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:53.953286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerStarted","Data":"68ee044da646b43d3f3a2eb92802d88c21b485050044d9b6b7a2b349ed8d22c1"}
Aug 03 16:06:53.953672 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:53.953327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerStarted","Data":"a0dcd4f74e64c7f54b93a0014c73bec8d94b65de6960a4dbaee7fd75f3f22d95"}
Aug 03 16:06:53.953672 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:53.953385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerStarted","Data":"50217c60bfa7f3b13a70354e0774ae31634223789c09ad57399c4498fe310b90"}
Aug 03 16:06:53.953672 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:53.953421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0a56f11f-47f6-440f-9071-f0fd0d910489","Type":"ContainerStarted","Data":"8bbcca39885ebea14cf68e64680048698cd4cfedced3e9307f2b203220e13ebd"}
Aug 03 16:06:53.982374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:53.982322    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.959714462 podStartE2EDuration="2.982306832s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.59237ms" isStatefulPod=false podCreationTimestamp="2026-08-03 16:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:06:53.980337184 +0000 UTC m=+156.247153397" watchObservedRunningTime="2026-08-03 16:06:53.982306832 +0000 UTC m=+156.249122838"
Aug 03 16:06:57.318652 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:06:57.318611    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:07:52.318025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:07:52.317993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:07:52.332562 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:07:52.332535    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:07:53.167529 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:07:53.167499    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 03 16:09:18.105037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:09:18.105009    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:09:18.106311 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:09:18.106284    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:09:18.111709 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:09:18.111685    2564 kubelet.go:1648] "Image garbage collection succeeded"
Aug 03 16:10:11.057916 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.057832    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"]
Aug 03 16:10:11.060858 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.060841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.064159 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.064134    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 03 16:10:11.064299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.064261    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6rvp9\""
Aug 03 16:10:11.064367 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.064343    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 03 16:10:11.064807 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.064788    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 03 16:10:11.064897 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.064869    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 03 16:10:11.078402 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.078376    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"]
Aug 03 16:10:11.230804 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.230768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/922e9513-26ab-4ed4-8892-86b775b00ad0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.230804 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.230806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/922e9513-26ab-4ed4-8892-86b775b00ad0-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.231062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.230826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/922e9513-26ab-4ed4-8892-86b775b00ad0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.231062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.230842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922e9513-26ab-4ed4-8892-86b775b00ad0-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.231062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.230961    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4555b\" (UniqueName: \"kubernetes.io/projected/922e9513-26ab-4ed4-8892-86b775b00ad0-kube-api-access-4555b\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.331986 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.331951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/922e9513-26ab-4ed4-8892-86b775b00ad0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.332168 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.331993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/922e9513-26ab-4ed4-8892-86b775b00ad0-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.332168 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.332015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/922e9513-26ab-4ed4-8892-86b775b00ad0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.332168 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.332037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922e9513-26ab-4ed4-8892-86b775b00ad0-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.332168 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.332091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4555b\" (UniqueName: \"kubernetes.io/projected/922e9513-26ab-4ed4-8892-86b775b00ad0-kube-api-access-4555b\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.332376 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.332354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/922e9513-26ab-4ed4-8892-86b775b00ad0-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.334248 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.334227    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922e9513-26ab-4ed4-8892-86b775b00ad0-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.334488 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.334471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/922e9513-26ab-4ed4-8892-86b775b00ad0-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.334554 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.334506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/922e9513-26ab-4ed4-8892-86b775b00ad0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.340196 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.340170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4555b\" (UniqueName: \"kubernetes.io/projected/922e9513-26ab-4ed4-8892-86b775b00ad0-kube-api-access-4555b\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-jd9rw\" (UID: \"922e9513-26ab-4ed4-8892-86b775b00ad0\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.370540 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.370519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:11.502811 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.502645    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"]
Aug 03 16:10:11.505457 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:10:11.505429    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922e9513_26ab_4ed4_8892_86b775b00ad0.slice/crio-706dceb72c29e2976f657d52429e1a234df7ef04b61cf282ffff059841b12ce9 WatchSource:0}: Error finding container 706dceb72c29e2976f657d52429e1a234df7ef04b61cf282ffff059841b12ce9: Status 404 returned error can't find the container with id 706dceb72c29e2976f657d52429e1a234df7ef04b61cf282ffff059841b12ce9
Aug 03 16:10:11.507113 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.507096    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:10:11.611680 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:11.611598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw" event={"ID":"922e9513-26ab-4ed4-8892-86b775b00ad0","Type":"ContainerStarted","Data":"706dceb72c29e2976f657d52429e1a234df7ef04b61cf282ffff059841b12ce9"}
Aug 03 16:10:15.628888 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:15.628851    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw" event={"ID":"922e9513-26ab-4ed4-8892-86b775b00ad0","Type":"ContainerStarted","Data":"c2d7b00c2057e7c82d656237f35742ecd443af8387b7ef4ed993920e6fc5a268"}
Aug 03 16:10:16.634029 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:16.633997    2564 generic.go:356] "Generic (PLEG): container finished" podID="922e9513-26ab-4ed4-8892-86b775b00ad0" containerID="c2d7b00c2057e7c82d656237f35742ecd443af8387b7ef4ed993920e6fc5a268" exitCode=0
Aug 03 16:10:16.634404 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:16.634082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw" event={"ID":"922e9513-26ab-4ed4-8892-86b775b00ad0","Type":"ContainerDied","Data":"c2d7b00c2057e7c82d656237f35742ecd443af8387b7ef4ed993920e6fc5a268"}
Aug 03 16:10:17.641032 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:17.640997    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw" event={"ID":"922e9513-26ab-4ed4-8892-86b775b00ad0","Type":"ContainerStarted","Data":"11f2fb01cd2b5a5eb4af87a8c1549bfc88304bb92d7159bab06b512fceb7ffd9"}
Aug 03 16:10:17.641527 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:17.641129    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:17.666962 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:17.666901    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw" podStartSLOduration=2.033205856 podStartE2EDuration="6.666884613s" totalImagesPullingTime="3.551428054s" totalInitContainerRuntime="1.082250703s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:17.664849079 +0000 UTC m=+359.931665086" watchObservedRunningTime="2026-08-03 16:10:17.666884613 +0000 UTC m=+359.933700622"
Aug 03 16:10:27.177473 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.177434    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fbcc56d8f-9w6v5"]
Aug 03 16:10:27.190457 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.190429    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.191681 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.191654    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbcc56d8f-9w6v5"]
Aug 03 16:10:27.193185 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.193153    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 03 16:10:27.193312 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.193163    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 03 16:10:27.193312 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.193168    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 03 16:10:27.193467 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.193312    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 03 16:10:27.194545 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.194356    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 03 16:10:27.194545 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.194411    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 03 16:10:27.194545 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.194472    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5c8lp\""
Aug 03 16:10:27.194545 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.194494    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 03 16:10:27.202341 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.202317    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Aug 03 16:10:27.258544 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.258515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-service-ca\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.258688 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.258555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24cdd403-0b54-40d6-8522-65f12b3088fb-console-serving-cert\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.258688 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.258575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-trusted-ca-bundle\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.258688 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.258590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-oauth-serving-cert\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.258688 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.258626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24cdd403-0b54-40d6-8522-65f12b3088fb-console-oauth-config\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.258688 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.258669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-console-config\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.258876 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.258715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdjpq\" (UniqueName: \"kubernetes.io/projected/24cdd403-0b54-40d6-8522-65f12b3088fb-kube-api-access-rdjpq\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.359576 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.359536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdjpq\" (UniqueName: \"kubernetes.io/projected/24cdd403-0b54-40d6-8522-65f12b3088fb-kube-api-access-rdjpq\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.359726 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.359593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-service-ca\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.359726 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.359633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24cdd403-0b54-40d6-8522-65f12b3088fb-console-serving-cert\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.359726 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.359658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-trusted-ca-bundle\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.359726 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.359678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-oauth-serving-cert\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.359944 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.359838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24cdd403-0b54-40d6-8522-65f12b3088fb-console-oauth-config\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.359944 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.359891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-console-config\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.360589 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.360553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-service-ca\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.360699 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.360599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-trusted-ca-bundle\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.360699 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.360653    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-oauth-serving-cert\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.360837 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.360781    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24cdd403-0b54-40d6-8522-65f12b3088fb-console-config\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.362021 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.361993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24cdd403-0b54-40d6-8522-65f12b3088fb-console-serving-cert\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.362317 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.362299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24cdd403-0b54-40d6-8522-65f12b3088fb-console-oauth-config\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.368553 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.368534    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdjpq\" (UniqueName: \"kubernetes.io/projected/24cdd403-0b54-40d6-8522-65f12b3088fb-kube-api-access-rdjpq\") pod \"console-fbcc56d8f-9w6v5\" (UID: \"24cdd403-0b54-40d6-8522-65f12b3088fb\") " pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.505289 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.505212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:27.641457 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.641434    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbcc56d8f-9w6v5"]
Aug 03 16:10:27.643999 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:10:27.643971    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cdd403_0b54_40d6_8522_65f12b3088fb.slice/crio-1fe493855453a1efccb2dddec661d3c37816513726879cba3b2d0236e852ccfd WatchSource:0}: Error finding container 1fe493855453a1efccb2dddec661d3c37816513726879cba3b2d0236e852ccfd: Status 404 returned error can't find the container with id 1fe493855453a1efccb2dddec661d3c37816513726879cba3b2d0236e852ccfd
Aug 03 16:10:27.675322 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:27.675295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbcc56d8f-9w6v5" event={"ID":"24cdd403-0b54-40d6-8522-65f12b3088fb","Type":"ContainerStarted","Data":"1fe493855453a1efccb2dddec661d3c37816513726879cba3b2d0236e852ccfd"}
Aug 03 16:10:28.647009 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:28.646977    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-jd9rw"
Aug 03 16:10:28.680692 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:28.680658    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbcc56d8f-9w6v5" event={"ID":"24cdd403-0b54-40d6-8522-65f12b3088fb","Type":"ContainerStarted","Data":"1071ec8bb93c8f7b484bcf1aec60dffe8a7efd3af6a595895b57b3b9db32b447"}
Aug 03 16:10:28.698227 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:28.698152    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fbcc56d8f-9w6v5" podStartSLOduration=1.69813423 podStartE2EDuration="1.69813423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:28.697412874 +0000 UTC m=+370.964228881" watchObservedRunningTime="2026-08-03 16:10:28.69813423 +0000 UTC m=+370.964950241"
Aug 03 16:10:31.256938 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.256905    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qz2d8"]
Aug 03 16:10:31.259977 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.259958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:31.262895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.262870    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Aug 03 16:10:31.262999 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.262904    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7mpnw\""
Aug 03 16:10:31.270028 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.270009    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qz2d8"]
Aug 03 16:10:31.290573 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.290550    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5l8z\" (UniqueName: \"kubernetes.io/projected/c7324a4a-a439-4830-966f-41bcb5d1023d-kube-api-access-j5l8z\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:31.290664 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.290589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:31.391180 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.391146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5l8z\" (UniqueName: \"kubernetes.io/projected/c7324a4a-a439-4830-966f-41bcb5d1023d-kube-api-access-j5l8z\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:31.391349 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.391188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:31.391349 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:10:31.391286    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Aug 03 16:10:31.391349 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:10:31.391337    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert podName:c7324a4a-a439-4830-966f-41bcb5d1023d nodeName:}" failed. No retries permitted until 2026-08-03 16:10:31.891320205 +0000 UTC m=+374.158136191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert") pod "odh-model-controller-7998c67dc4-qz2d8" (UID: "c7324a4a-a439-4830-966f-41bcb5d1023d") : secret "odh-model-controller-webhook-cert" not found
Aug 03 16:10:31.402885 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.402862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5l8z\" (UniqueName: \"kubernetes.io/projected/c7324a4a-a439-4830-966f-41bcb5d1023d-kube-api-access-j5l8z\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:31.893965 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:31.893930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:31.894161 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:10:31.894112    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Aug 03 16:10:31.894233 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:10:31.894205    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert podName:c7324a4a-a439-4830-966f-41bcb5d1023d nodeName:}" failed. No retries permitted until 2026-08-03 16:10:32.894182136 +0000 UTC m=+375.160998134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert") pod "odh-model-controller-7998c67dc4-qz2d8" (UID: "c7324a4a-a439-4830-966f-41bcb5d1023d") : secret "odh-model-controller-webhook-cert" not found
Aug 03 16:10:32.902860 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:32.902827    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:32.905275 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:32.905249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c7324a4a-a439-4830-966f-41bcb5d1023d-cert\") pod \"odh-model-controller-7998c67dc4-qz2d8\" (UID: \"c7324a4a-a439-4830-966f-41bcb5d1023d\") " pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:33.073315 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:33.073270    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:33.208708 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:33.208676    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qz2d8"]
Aug 03 16:10:33.212109 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:10:33.212079    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7324a4a_a439_4830_966f_41bcb5d1023d.slice/crio-45daefb24e6b44a0a819a2e03350ec05d14492872e4f8b551023f951b7aee5d3 WatchSource:0}: Error finding container 45daefb24e6b44a0a819a2e03350ec05d14492872e4f8b551023f951b7aee5d3: Status 404 returned error can't find the container with id 45daefb24e6b44a0a819a2e03350ec05d14492872e4f8b551023f951b7aee5d3
Aug 03 16:10:33.699548 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:33.699511    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8" event={"ID":"c7324a4a-a439-4830-966f-41bcb5d1023d","Type":"ContainerStarted","Data":"45daefb24e6b44a0a819a2e03350ec05d14492872e4f8b551023f951b7aee5d3"}
Aug 03 16:10:36.718342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:36.718248    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8" event={"ID":"c7324a4a-a439-4830-966f-41bcb5d1023d","Type":"ContainerStarted","Data":"af82ee5fd7258bd1a07addcf5a5819799a5a4c9e069b6efbbee47b1efd9a48da"}
Aug 03 16:10:36.718342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:36.718318    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:36.742617 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:36.742553    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8" podStartSLOduration=2.500057038 podStartE2EDuration="5.742537749s" totalImagesPullingTime="3.242480711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:36.741153394 +0000 UTC m=+379.007969402" watchObservedRunningTime="2026-08-03 16:10:36.742537749 +0000 UTC m=+379.009353747"
Aug 03 16:10:37.505588 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:37.505554    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:37.505802 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:37.505620    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:37.510941 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:37.510919    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:37.726612 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:37.726579    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fbcc56d8f-9w6v5"
Aug 03 16:10:41.005182 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.005151    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"]
Aug 03 16:10:41.008171 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.008155    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.010868 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.010842    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 03 16:10:41.010997 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.010893    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 03 16:10:41.012135 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.012115    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 03 16:10:41.012135 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.012126    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:10:41.012534 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.012473    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 03 16:10:41.012616 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.012532    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nhq9j\""
Aug 03 16:10:41.019703 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.019681    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"]
Aug 03 16:10:41.070909 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.070873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8c8103b-b410-4f60-9371-66a280a69600-cert\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.071094 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.070928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8c8103b-b410-4f60-9371-66a280a69600-metrics-cert\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.071094 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.071005    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc8n6\" (UniqueName: \"kubernetes.io/projected/b8c8103b-b410-4f60-9371-66a280a69600-kube-api-access-zc8n6\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.071094 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.071036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b8c8103b-b410-4f60-9371-66a280a69600-manager-config\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.172197 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.172165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8c8103b-b410-4f60-9371-66a280a69600-cert\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.172377 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.172204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8c8103b-b410-4f60-9371-66a280a69600-metrics-cert\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.172377 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.172243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc8n6\" (UniqueName: \"kubernetes.io/projected/b8c8103b-b410-4f60-9371-66a280a69600-kube-api-access-zc8n6\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.172377 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.172268    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b8c8103b-b410-4f60-9371-66a280a69600-manager-config\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.173027 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.173000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b8c8103b-b410-4f60-9371-66a280a69600-manager-config\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.174709 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.174688    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8c8103b-b410-4f60-9371-66a280a69600-cert\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.174973 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.174951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8c8103b-b410-4f60-9371-66a280a69600-metrics-cert\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.184774 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.184753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc8n6\" (UniqueName: \"kubernetes.io/projected/b8c8103b-b410-4f60-9371-66a280a69600-kube-api-access-zc8n6\") pod \"lws-controller-manager-7596f8d454-8r7jv\" (UID: \"b8c8103b-b410-4f60-9371-66a280a69600\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.318092 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.318058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:41.463782 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.463757    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"]
Aug 03 16:10:41.464838 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:10:41.464808    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c8103b_b410_4f60_9371_66a280a69600.slice/crio-862dbf0f4a8fae84bad58bd1e485aaea2b54704f510e31aaa95686cf2847f8b4 WatchSource:0}: Error finding container 862dbf0f4a8fae84bad58bd1e485aaea2b54704f510e31aaa95686cf2847f8b4: Status 404 returned error can't find the container with id 862dbf0f4a8fae84bad58bd1e485aaea2b54704f510e31aaa95686cf2847f8b4
Aug 03 16:10:41.636334 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.636268    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"]
Aug 03 16:10:41.640544 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.640527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.648950 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.648928    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 03 16:10:41.651320 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.651302    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 03 16:10:41.651415 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.651307    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-srfwq\""
Aug 03 16:10:41.651817 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.651801    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 03 16:10:41.651915 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.651802    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 03 16:10:41.661661 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.661643    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"]
Aug 03 16:10:41.736374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.736340    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv" event={"ID":"b8c8103b-b410-4f60-9371-66a280a69600","Type":"ContainerStarted","Data":"862dbf0f4a8fae84bad58bd1e485aaea2b54704f510e31aaa95686cf2847f8b4"}
Aug 03 16:10:41.776942 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.776919    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cr8q\" (UniqueName: \"kubernetes.io/projected/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-kube-api-access-2cr8q\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.777045 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.776967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-tls-certs\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.777045 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.777029    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-tmp\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.877778 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.877726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-tmp\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.877924 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.877795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cr8q\" (UniqueName: \"kubernetes.io/projected/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-kube-api-access-2cr8q\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.877924 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.877874    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-tls-certs\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.880288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.880266    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-tmp\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.880426 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.880408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-tls-certs\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.887370 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.887332    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cr8q\" (UniqueName: \"kubernetes.io/projected/607ce5d7-90c3-4174-ad8d-8ab698fea2fe-kube-api-access-2cr8q\") pod \"kube-auth-proxy-845c84c75b-q96jk\" (UID: \"607ce5d7-90c3-4174-ad8d-8ab698fea2fe\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:41.950895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:41.950869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"
Aug 03 16:10:42.074185 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:42.074160    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-845c84c75b-q96jk"]
Aug 03 16:10:42.076452 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:10:42.076427    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod607ce5d7_90c3_4174_ad8d_8ab698fea2fe.slice/crio-ce49efceef871031489331fb9f0e77a73e655a51c94cda79f6dd603fef89ddb0 WatchSource:0}: Error finding container ce49efceef871031489331fb9f0e77a73e655a51c94cda79f6dd603fef89ddb0: Status 404 returned error can't find the container with id ce49efceef871031489331fb9f0e77a73e655a51c94cda79f6dd603fef89ddb0
Aug 03 16:10:42.741132 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:42.741099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk" event={"ID":"607ce5d7-90c3-4174-ad8d-8ab698fea2fe","Type":"ContainerStarted","Data":"ce49efceef871031489331fb9f0e77a73e655a51c94cda79f6dd603fef89ddb0"}
Aug 03 16:10:46.762159 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:46.762123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk" event={"ID":"607ce5d7-90c3-4174-ad8d-8ab698fea2fe","Type":"ContainerStarted","Data":"9db683df5a2dbe7453bace106f522037bf5b3a7b4d334b84826a4c583f64c0cd"}
Aug 03 16:10:46.763601 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:46.763578    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv" event={"ID":"b8c8103b-b410-4f60-9371-66a280a69600","Type":"ContainerStarted","Data":"31efa6d0bcf2e2c3dd31171f44839763fc07f72c5a9d3a0ae68a5c61396ba86e"}
Aug 03 16:10:46.763746 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:46.763718    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:10:46.784413 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:46.784367    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-845c84c75b-q96jk" podStartSLOduration=1.773553757 podStartE2EDuration="5.784354246s" totalImagesPullingTime="4.010800489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:46.782585532 +0000 UTC m=+389.049401538" watchObservedRunningTime="2026-08-03 16:10:46.784354246 +0000 UTC m=+389.051170252"
Aug 03 16:10:46.800776 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:46.800718    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv" podStartSLOduration=2.184526739 podStartE2EDuration="6.800704981s" totalImagesPullingTime="4.616178242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:46.799220912 +0000 UTC m=+389.066036922" watchObservedRunningTime="2026-08-03 16:10:46.800704981 +0000 UTC m=+389.067520987"
Aug 03 16:10:47.724120 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:47.724092    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qz2d8"
Aug 03 16:10:57.771855 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:10:57.771825    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-8r7jv"
Aug 03 16:11:52.859950 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:52.859870    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"]
Aug 03 16:11:52.861919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:52.861904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:11:52.864625 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:52.864604    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 03 16:11:52.865760 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:52.865726    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 03 16:11:52.865760 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:52.865748    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zjr4q\""
Aug 03 16:11:52.881107 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:52.881083    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"]
Aug 03 16:11:53.007577 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:53.007539    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsm88\" (UniqueName: \"kubernetes.io/projected/37e0e639-ceb0-43ef-ba54-528affcc3a92-kube-api-access-gsm88\") pod \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" (UID: \"37e0e639-ceb0-43ef-ba54-528affcc3a92\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:11:53.108228 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:53.108192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsm88\" (UniqueName: \"kubernetes.io/projected/37e0e639-ceb0-43ef-ba54-528affcc3a92-kube-api-access-gsm88\") pod \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" (UID: \"37e0e639-ceb0-43ef-ba54-528affcc3a92\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:11:53.117812 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:53.117755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsm88\" (UniqueName: \"kubernetes.io/projected/37e0e639-ceb0-43ef-ba54-528affcc3a92-kube-api-access-gsm88\") pod \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" (UID: \"37e0e639-ceb0-43ef-ba54-528affcc3a92\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:11:53.171382 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:53.171352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:11:53.303810 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:53.303782    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"]
Aug 03 16:11:53.306107 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:11:53.306082    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e0e639_ceb0_43ef_ba54_528affcc3a92.slice/crio-4d1f93024af7f4ca090307e4c970e9ebe5ab54f7cef7118bc8a5c847361da7f1 WatchSource:0}: Error finding container 4d1f93024af7f4ca090307e4c970e9ebe5ab54f7cef7118bc8a5c847361da7f1: Status 404 returned error can't find the container with id 4d1f93024af7f4ca090307e4c970e9ebe5ab54f7cef7118bc8a5c847361da7f1
Aug 03 16:11:54.028604 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:54.028571    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc" event={"ID":"37e0e639-ceb0-43ef-ba54-528affcc3a92","Type":"ContainerStarted","Data":"4d1f93024af7f4ca090307e4c970e9ebe5ab54f7cef7118bc8a5c847361da7f1"}
Aug 03 16:11:55.980948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:55.980909    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"]
Aug 03 16:11:55.984109 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:55.984086    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:55.986897 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:55.986873    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Aug 03 16:11:55.987006 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:55.986875    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-psr2n\""
Aug 03 16:11:55.987138 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:55.987124    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Aug 03 16:11:55.997378 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:55.997357    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"]
Aug 03 16:11:56.034175 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.034143    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/32938926-5f06-423d-918f-bded0ddf7d64-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.034338 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.034198    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njgjd\" (UniqueName: \"kubernetes.io/projected/32938926-5f06-423d-918f-bded0ddf7d64-kube-api-access-njgjd\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.034338 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.034223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32938926-5f06-423d-918f-bded0ddf7d64-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.037850 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.037820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc" event={"ID":"37e0e639-ceb0-43ef-ba54-528affcc3a92","Type":"ContainerStarted","Data":"860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273"}
Aug 03 16:11:56.037977 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.037949    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:11:56.063964 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.063916    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc" podStartSLOduration=1.532790399 podStartE2EDuration="4.063901845s" totalImagesPullingTime="2.531111446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:11:56.06290609 +0000 UTC m=+458.329722109" watchObservedRunningTime="2026-08-03 16:11:56.063901845 +0000 UTC m=+458.330717851"
Aug 03 16:11:56.134728 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.134649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32938926-5f06-423d-918f-bded0ddf7d64-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.134912 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.134769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/32938926-5f06-423d-918f-bded0ddf7d64-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.134912 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:11:56.134802    2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Aug 03 16:11:56.134912 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.134831    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njgjd\" (UniqueName: \"kubernetes.io/projected/32938926-5f06-423d-918f-bded0ddf7d64-kube-api-access-njgjd\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.134912 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:11:56.134875    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32938926-5f06-423d-918f-bded0ddf7d64-plugin-serving-cert podName:32938926-5f06-423d-918f-bded0ddf7d64 nodeName:}" failed. No retries permitted until 2026-08-03 16:11:56.634853192 +0000 UTC m=+458.901669183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/32938926-5f06-423d-918f-bded0ddf7d64-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-l2vkb" (UID: "32938926-5f06-423d-918f-bded0ddf7d64") : secret "plugin-serving-cert" not found
Aug 03 16:11:56.135384 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.135363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/32938926-5f06-423d-918f-bded0ddf7d64-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.144029 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.144008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njgjd\" (UniqueName: \"kubernetes.io/projected/32938926-5f06-423d-918f-bded0ddf7d64-kube-api-access-njgjd\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.638544 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.638509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32938926-5f06-423d-918f-bded0ddf7d64-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.641013 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.640992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32938926-5f06-423d-918f-bded0ddf7d64-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-l2vkb\" (UID: \"32938926-5f06-423d-918f-bded0ddf7d64\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:56.901058 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:56.900968    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"
Aug 03 16:11:57.033587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:57.033554    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb"]
Aug 03 16:11:57.036964 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:11:57.036941    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32938926_5f06_423d_918f_bded0ddf7d64.slice/crio-e81c5a87c5d73d039c572a9f9b7e114bfc765436d9302f0addc5b6baffd27332 WatchSource:0}: Error finding container e81c5a87c5d73d039c572a9f9b7e114bfc765436d9302f0addc5b6baffd27332: Status 404 returned error can't find the container with id e81c5a87c5d73d039c572a9f9b7e114bfc765436d9302f0addc5b6baffd27332
Aug 03 16:11:57.043523 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:11:57.043495    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb" event={"ID":"32938926-5f06-423d-918f-bded0ddf7d64","Type":"ContainerStarted","Data":"e81c5a87c5d73d039c572a9f9b7e114bfc765436d9302f0addc5b6baffd27332"}
Aug 03 16:12:02.070867 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:02.070828    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb" event={"ID":"32938926-5f06-423d-918f-bded0ddf7d64","Type":"ContainerStarted","Data":"7308045e092f0ae958868ebf31d1b0218b253c6529ffe1b3d524398b8ff45b04"}
Aug 03 16:12:02.089597 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:02.089554    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-l2vkb" podStartSLOduration=2.409066491 podStartE2EDuration="7.089541655s" totalImagesPullingTime="4.680475164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:02.087895862 +0000 UTC m=+464.354711872" watchObservedRunningTime="2026-08-03 16:12:02.089541655 +0000 UTC m=+464.356357661"
Aug 03 16:12:06.007961 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.007930    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"]
Aug 03 16:12:06.008374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.008119    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" containerName="manager" containerID="cri-o://860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273" gracePeriod=2
Aug 03 16:12:06.009812 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.009786    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:06.028242 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.028218    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"]
Aug 03 16:12:06.033784 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:12:06.033730    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37e0e639-ceb0-43ef-ba54-528affcc3a92\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-pw6qc\": pods \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:06.038197 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:12:06.038164    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:06.041672 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.041649    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"]
Aug 03 16:12:06.042002 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.041986    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" containerName="manager"
Aug 03 16:12:06.042077 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.042004    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" containerName="manager"
Aug 03 16:12:06.042135 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.042096    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" containerName="manager"
Aug 03 16:12:06.044941 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.044923    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"
Aug 03 16:12:06.047334 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:12:06.047304    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:06.057632 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.057611    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"]
Aug 03 16:12:06.118991 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.118959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxh4g\" (UniqueName: \"kubernetes.io/projected/0e66f32c-ba10-4836-871f-a132ee22a26c-kube-api-access-mxh4g\") pod \"limitador-operator-controller-manager-6c549b6d85-278cz\" (UID: \"0e66f32c-ba10-4836-871f-a132ee22a26c\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"
Aug 03 16:12:06.220040 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.220010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxh4g\" (UniqueName: \"kubernetes.io/projected/0e66f32c-ba10-4836-871f-a132ee22a26c-kube-api-access-mxh4g\") pod \"limitador-operator-controller-manager-6c549b6d85-278cz\" (UID: \"0e66f32c-ba10-4836-871f-a132ee22a26c\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"
Aug 03 16:12:06.229977 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.229947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxh4g\" (UniqueName: \"kubernetes.io/projected/0e66f32c-ba10-4836-871f-a132ee22a26c-kube-api-access-mxh4g\") pod \"limitador-operator-controller-manager-6c549b6d85-278cz\" (UID: \"0e66f32c-ba10-4836-871f-a132ee22a26c\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"
Aug 03 16:12:06.246886 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.246866    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:06.321057 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.321018    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsm88\" (UniqueName: \"kubernetes.io/projected/37e0e639-ceb0-43ef-ba54-528affcc3a92-kube-api-access-gsm88\") pod \"37e0e639-ceb0-43ef-ba54-528affcc3a92\" (UID: \"37e0e639-ceb0-43ef-ba54-528affcc3a92\") "
Aug 03 16:12:06.323152 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.323131    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37e0e639-ceb0-43ef-ba54-528affcc3a92-kube-api-access-gsm88" (OuterVolumeSpecName: "kube-api-access-gsm88") pod "37e0e639-ceb0-43ef-ba54-528affcc3a92" (UID: "37e0e639-ceb0-43ef-ba54-528affcc3a92"). InnerVolumeSpecName "kube-api-access-gsm88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:12:06.386325 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.386296    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"
Aug 03 16:12:06.421831 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.421792    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsm88\" (UniqueName: \"kubernetes.io/projected/37e0e639-ceb0-43ef-ba54-528affcc3a92-kube-api-access-gsm88\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:12:06.517803 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:06.517779    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"]
Aug 03 16:12:06.519811 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:12:06.519773    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e66f32c_ba10_4836_871f_a132ee22a26c.slice/crio-a0df4ac7bff3f9984b15929a7df6a9440455ca9b5be5025eeda391f3adb07b10 WatchSource:0}: Error finding container a0df4ac7bff3f9984b15929a7df6a9440455ca9b5be5025eeda391f3adb07b10: Status 404 returned error can't find the container with id a0df4ac7bff3f9984b15929a7df6a9440455ca9b5be5025eeda391f3adb07b10
Aug 03 16:12:07.094892 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.094862    2564 generic.go:356] "Generic (PLEG): container finished" podID="37e0e639-ceb0-43ef-ba54-528affcc3a92" containerID="860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273" exitCode=0
Aug 03 16:12:07.095328 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.094917    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:07.095328 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.094970    2564 scope.go:117] "RemoveContainer" containerID="860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273"
Aug 03 16:12:07.096758 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.096714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz" event={"ID":"0e66f32c-ba10-4836-871f-a132ee22a26c","Type":"ContainerStarted","Data":"ba35e09e9b6d94e3cf46860ea6f6169a1e36ca01c642a0c1df046e79d1443474"}
Aug 03 16:12:07.096878 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.096814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz" event={"ID":"0e66f32c-ba10-4836-871f-a132ee22a26c","Type":"ContainerStarted","Data":"a0df4ac7bff3f9984b15929a7df6a9440455ca9b5be5025eeda391f3adb07b10"}
Aug 03 16:12:07.096878 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.096859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"
Aug 03 16:12:07.103589 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.103567    2564 scope.go:117] "RemoveContainer" containerID="860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273"
Aug 03 16:12:07.103858 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:12:07.103834    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273\": container with ID starting with 860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273 not found: ID does not exist" containerID="860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273"
Aug 03 16:12:07.103927 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.103869    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273"} err="failed to get container status \"860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273\": rpc error: code = NotFound desc = could not find container \"860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273\": container with ID starting with 860af2ca68177090de7cdf644e0ea1641c639576b7b7e6d917e8471169f94273 not found: ID does not exist"
Aug 03 16:12:07.116352 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:07.116311    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz" podStartSLOduration=1.116297461 podStartE2EDuration="1.116297461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:07.114987591 +0000 UTC m=+469.381803600" watchObservedRunningTime="2026-08-03 16:12:07.116297461 +0000 UTC m=+469.383113468"
Aug 03 16:12:07.117172 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:12:07.117149    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:08.241770 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:12:08.241717    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-pw6qc\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-pw6qc"
Aug 03 16:12:08.243041 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:08.243016    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37e0e639-ceb0-43ef-ba54-528affcc3a92" path="/var/lib/kubelet/pods/37e0e639-ceb0-43ef-ba54-528affcc3a92/volumes"
Aug 03 16:12:18.104244 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:18.104213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-278cz"
Aug 03 16:12:47.053711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.053678    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:12:47.056745 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.056718    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.059314 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.059295    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 03 16:12:47.069258 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.069240    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:12:47.083606 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.083582    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:12:47.155664 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.155631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c33cda60-e479-4f41-b5f8-9f7887908846-config-file\") pod \"limitador-limitador-85d9db4d5d-kjlq6\" (UID: \"c33cda60-e479-4f41-b5f8-9f7887908846\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.155664 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.155665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cvsn\" (UniqueName: \"kubernetes.io/projected/c33cda60-e479-4f41-b5f8-9f7887908846-kube-api-access-6cvsn\") pod \"limitador-limitador-85d9db4d5d-kjlq6\" (UID: \"c33cda60-e479-4f41-b5f8-9f7887908846\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.257058 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.257020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c33cda60-e479-4f41-b5f8-9f7887908846-config-file\") pod \"limitador-limitador-85d9db4d5d-kjlq6\" (UID: \"c33cda60-e479-4f41-b5f8-9f7887908846\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.257239 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.257068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cvsn\" (UniqueName: \"kubernetes.io/projected/c33cda60-e479-4f41-b5f8-9f7887908846-kube-api-access-6cvsn\") pod \"limitador-limitador-85d9db4d5d-kjlq6\" (UID: \"c33cda60-e479-4f41-b5f8-9f7887908846\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.257836 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.257806    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c33cda60-e479-4f41-b5f8-9f7887908846-config-file\") pod \"limitador-limitador-85d9db4d5d-kjlq6\" (UID: \"c33cda60-e479-4f41-b5f8-9f7887908846\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.265496 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.265468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cvsn\" (UniqueName: \"kubernetes.io/projected/c33cda60-e479-4f41-b5f8-9f7887908846-kube-api-access-6cvsn\") pod \"limitador-limitador-85d9db4d5d-kjlq6\" (UID: \"c33cda60-e479-4f41-b5f8-9f7887908846\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.366430 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.366396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:47.496826 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:47.496794    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:12:47.500884 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:12:47.500854    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc33cda60_e479_4f41_b5f8_9f7887908846.slice/crio-abd550588676a67def8c570954149a20f942c97e87c99c8ead388a231299025c WatchSource:0}: Error finding container abd550588676a67def8c570954149a20f942c97e87c99c8ead388a231299025c: Status 404 returned error can't find the container with id abd550588676a67def8c570954149a20f942c97e87c99c8ead388a231299025c
Aug 03 16:12:48.265844 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:48.265808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6" event={"ID":"c33cda60-e479-4f41-b5f8-9f7887908846","Type":"ContainerStarted","Data":"abd550588676a67def8c570954149a20f942c97e87c99c8ead388a231299025c"}
Aug 03 16:12:49.271851 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:49.271746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6" event={"ID":"c33cda60-e479-4f41-b5f8-9f7887908846","Type":"ContainerStarted","Data":"1e9bf9dbf9e13b48b5870dfb96dade7dd216038fc063148af53bf1e4bda7525a"}
Aug 03 16:12:49.272204 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:49.271866    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:12:49.290925 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:12:49.290880    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6" podStartSLOduration=0.771247787 podStartE2EDuration="2.290865045s" totalImagesPullingTime="1.519617258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:49.290281053 +0000 UTC m=+511.557097062" watchObservedRunningTime="2026-08-03 16:12:49.290865045 +0000 UTC m=+511.557681051"
Aug 03 16:13:00.276907 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:13:00.276879    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"
Aug 03 16:13:55.409513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:13:55.409481    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:14:10.786045 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:10.786006    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:14:18.131042 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:18.131014    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:14:18.131765 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:18.131014    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:14:20.184696 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:20.184651    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:14:29.999699 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:29.999658    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:14:52.981748 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:52.981690    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:14:58.735610 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.735575    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"]
Aug 03 16:14:58.738915 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.738893    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.741720 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.741696    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 03 16:14:58.742986 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.742961    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Aug 03 16:14:58.743088 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.742961    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 03 16:14:58.743088 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.743006    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6svrh\""
Aug 03 16:14:58.749887 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.749859    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"]
Aug 03 16:14:58.881132 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.881091    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.881132 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.881137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.881342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.881169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.881342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.881256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a421d1-78f8-478a-9764-8070dd403b14-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.881342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.881322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.881447 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.881348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2hgt\" (UniqueName: \"kubernetes.io/projected/37a421d1-78f8-478a-9764-8070dd403b14-kube-api-access-l2hgt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.982513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.982479    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.982513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.982515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2hgt\" (UniqueName: \"kubernetes.io/projected/37a421d1-78f8-478a-9764-8070dd403b14-kube-api-access-l2hgt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.982704 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.982533    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.982704 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.982568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.982704 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.982604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.982704 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.982662    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a421d1-78f8-478a-9764-8070dd403b14-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.983031 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.983000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.983097 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.983078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.983133 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.983094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.985020 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.984997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a421d1-78f8-478a-9764-8070dd403b14-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.985184 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.985168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a421d1-78f8-478a-9764-8070dd403b14-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:58.993145 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:58.993091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2hgt\" (UniqueName: \"kubernetes.io/projected/37a421d1-78f8-478a-9764-8070dd403b14-kube-api-access-l2hgt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zhmbp\" (UID: \"37a421d1-78f8-478a-9764-8070dd403b14\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:59.050291 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:59.050243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:14:59.187256 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:59.187231    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"]
Aug 03 16:14:59.189438 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:14:59.189401    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a421d1_78f8_478a_9764_8070dd403b14.slice/crio-0d7747d10dcff52b725d14183a53838ed49864fbe84b820278e981c3b428637d WatchSource:0}: Error finding container 0d7747d10dcff52b725d14183a53838ed49864fbe84b820278e981c3b428637d: Status 404 returned error can't find the container with id 0d7747d10dcff52b725d14183a53838ed49864fbe84b820278e981c3b428637d
Aug 03 16:14:59.801725 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:14:59.801683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp" event={"ID":"37a421d1-78f8-478a-9764-8070dd403b14","Type":"ContainerStarted","Data":"0d7747d10dcff52b725d14183a53838ed49864fbe84b820278e981c3b428637d"}
Aug 03 16:15:00.794489 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:00.794405    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:15:00.807675 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:00.807644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp" event={"ID":"37a421d1-78f8-478a-9764-8070dd403b14","Type":"ContainerStarted","Data":"78668903500b5d80892878a3d6f1bd63ee49d354fbfb6434d6caea86ab75a0da"}
Aug 03 16:15:00.828344 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:00.828297    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp" podStartSLOduration=1.6311120799999999 podStartE2EDuration="2.828281893s" totalImagesPullingTime="1.197169813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:15:00.826348384 +0000 UTC m=+643.093164393" watchObservedRunningTime="2026-08-03 16:15:00.828281893 +0000 UTC m=+643.095097879"
Aug 03 16:15:09.050696 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.050648    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:15:09.051171 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.050721    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:15:09.063235 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.063212    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:15:09.547766 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.547718    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"]
Aug 03 16:15:09.551131 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.551111    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.553775 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.553713    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Aug 03 16:15:09.559886 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.559864    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"]
Aug 03 16:15:09.674095 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.674060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.674095 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.674099    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.674321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.674115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.674321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.674138    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbchw\" (UniqueName: \"kubernetes.io/projected/b3956d23-c892-4558-bc02-36deb60e0039-kube-api-access-dbchw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.674321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.674190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.674321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.674240    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3956d23-c892-4558-bc02-36deb60e0039-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.774854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.774815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.774854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.774856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3956d23-c892-4558-bc02-36deb60e0039-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.775059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.774965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.775059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.774995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.775059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.775016    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.775059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.775049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbchw\" (UniqueName: \"kubernetes.io/projected/b3956d23-c892-4558-bc02-36deb60e0039-kube-api-access-dbchw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.775361 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.775337    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.775433 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.775405    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.775433 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.775418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.777221 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.777203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3956d23-c892-4558-bc02-36deb60e0039-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.777589 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.777570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3956d23-c892-4558-bc02-36deb60e0039-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.784373 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.784347    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbchw\" (UniqueName: \"kubernetes.io/projected/b3956d23-c892-4558-bc02-36deb60e0039-kube-api-access-dbchw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwhft\" (UID: \"b3956d23-c892-4558-bc02-36deb60e0039\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.860491 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.860465    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zhmbp"
Aug 03 16:15:09.862760 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.862725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:09.995958 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:09.995937    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"]
Aug 03 16:15:09.998526 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:15:09.998501    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3956d23_c892_4558_bc02_36deb60e0039.slice/crio-13dadd612740a8547000b4be0aa84f68c243084844d8dfe51b294e2cb98020f6 WatchSource:0}: Error finding container 13dadd612740a8547000b4be0aa84f68c243084844d8dfe51b294e2cb98020f6: Status 404 returned error can't find the container with id 13dadd612740a8547000b4be0aa84f68c243084844d8dfe51b294e2cb98020f6
Aug 03 16:15:10.854417 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:10.854378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft" event={"ID":"b3956d23-c892-4558-bc02-36deb60e0039","Type":"ContainerStarted","Data":"13dadd612740a8547000b4be0aa84f68c243084844d8dfe51b294e2cb98020f6"}
Aug 03 16:15:12.013905 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:12.013877    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:15:12.866580 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:12.866541    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft" event={"ID":"b3956d23-c892-4558-bc02-36deb60e0039","Type":"ContainerStarted","Data":"0277bb25cd2d2015f92fcf45ba62041a116c2eca85bcf023721f8c8596d4ec7b"}
Aug 03 16:15:12.886374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:12.886329    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft" podStartSLOduration=1.869463517 podStartE2EDuration="3.886312488s" totalImagesPullingTime="2.016848971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:15:12.884911872 +0000 UTC m=+655.151727876" watchObservedRunningTime="2026-08-03 16:15:12.886312488 +0000 UTC m=+655.153128495"
Aug 03 16:15:19.863572 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:19.863537    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:19.863996 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:19.863669    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:19.875956 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:19.875932    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:15:19.907211 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:15:19.907182    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwhft"
Aug 03 16:16:27.600379 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:16:27.600305    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:16:37.591470 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:16:37.591430    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:16:49.099904 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:16:49.099867    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:18:28.303609 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:18:28.303573    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:18:38.098297 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:18:38.098263    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:19:18.162028 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:19:18.161949    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:19:18.163861 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:19:18.163836    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:19:38.495694 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:19:38.495659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:19:45.397482 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:19:45.397441    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:20:00.695150 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:20:00.695117    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:20:06.496838 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:20:06.496799    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:20:16.198071 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:20:16.198035    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:20:22.799197 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:20:22.799163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:20:36.993208 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:20:36.993169    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:20:42.308711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:20:42.308617    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:20:47.287931 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:20:47.287901    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:21:09.094330 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:21:09.094298    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:21:19.698805 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:21:19.698769    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:21:42.928523 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:21:42.928487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:21:53.604342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:21:53.604307    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:21:59.303542 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:21:59.303506    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:22:07.002285 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:22:07.002251    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:22:14.892401 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:22:14.892318    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:22:19.599766 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:22:19.599717    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:22:24.507204 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:22:24.507164    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:22:31.893903 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:22:31.893871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:22:44.206176 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:22:44.206137    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:22:51.198222 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:22:51.198186    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:23:02.199097 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:23:02.199055    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:23:07.091474 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:23:07.091441    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:23:13.489693 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:23:13.489659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:23:22.684392 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:23:22.684356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:23:28.996308 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:23:28.996272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:23:39.392164 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:23:39.392123    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:24:18.185219 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:24:18.185188    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:24:18.188070 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:24:18.188052    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:25:56.306221 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:25:56.306193    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:26:01.295975 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:26:01.295940    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:26:24.995134 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:26:24.995096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:26:36.296878 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:26:36.296846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:26:46.891634 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:26:46.891549    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:26:51.797386 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:26:51.797351    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:26:56.685603 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:26:56.685569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:27:06.987873 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:27:06.987837    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:27:12.584750 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:27:12.584703    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:27:22.191774 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:27:22.191717    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:27:27.894463 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:27:27.894429    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:27:37.485607 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:27:37.485572    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:27:43.694651 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:27:43.694617    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:27:53.292387 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:27:53.292350    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:28:17.792138 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:17.792046    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:28:23.183934 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:23.183902    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:28:32.454663 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.454626    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"]
Aug 03 16:28:32.458403 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.458377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.461188 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.461166    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ed4bc549-kserve-self-signed-certs\""
Aug 03 16:28:32.473708 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.473683    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"]
Aug 03 16:28:32.577104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.577069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-model-cache\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.577104 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.577103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-tmp-dir\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.577311 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.577156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f222c2-30a0-426c-aab9-861f68ee8000-tls-certs\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.577311 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.577173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-home\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.577311 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.577189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-dshm\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.577311 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.577206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wf8s\" (UniqueName: \"kubernetes.io/projected/92f222c2-30a0-426c-aab9-861f68ee8000-kube-api-access-8wf8s\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678323    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-model-cache\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678374 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678377    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-tmp-dir\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678607 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678456    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f222c2-30a0-426c-aab9-861f68ee8000-tls-certs\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678607 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-home\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678607 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-dshm\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678607 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wf8s\" (UniqueName: \"kubernetes.io/projected/92f222c2-30a0-426c-aab9-861f68ee8000-kube-api-access-8wf8s\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678888 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-model-cache\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.678945 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678906    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-home\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.679000 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.678982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-tmp-dir\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.680676 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.680659    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-dshm\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.681035 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.681019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f222c2-30a0-426c-aab9-861f68ee8000-tls-certs\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.687377 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.687350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wf8s\" (UniqueName: \"kubernetes.io/projected/92f222c2-30a0-426c-aab9-861f68ee8000-kube-api-access-8wf8s\") pod \"e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") " pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.767784 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.767673    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:32.896860 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:32.896837    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"]
Aug 03 16:28:32.899397 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:28:32.899366    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92f222c2_30a0_426c_aab9_861f68ee8000.slice/crio-dc1f7566f4f3688a7a91c51a4e2595cbb6673b47f688e0fb532595582e56df67 WatchSource:0}: Error finding container dc1f7566f4f3688a7a91c51a4e2595cbb6673b47f688e0fb532595582e56df67: Status 404 returned error can't find the container with id dc1f7566f4f3688a7a91c51a4e2595cbb6673b47f688e0fb532595582e56df67
Aug 03 16:28:33.201361 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:33.201325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h" event={"ID":"92f222c2-30a0-426c-aab9-861f68ee8000","Type":"ContainerStarted","Data":"1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24"}
Aug 03 16:28:33.201562 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:33.201371    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h" event={"ID":"92f222c2-30a0-426c-aab9-861f68ee8000","Type":"ContainerStarted","Data":"dc1f7566f4f3688a7a91c51a4e2595cbb6673b47f688e0fb532595582e56df67"}
Aug 03 16:28:33.221310 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:33.221261    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h" podStartSLOduration=1.221245781 podStartE2EDuration="1.221245781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:28:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:28:33.21883664 +0000 UTC m=+1455.485652647" watchObservedRunningTime="2026-08-03 16:28:33.221245781 +0000 UTC m=+1455.488061787"
Aug 03 16:28:35.403992 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:35.403957    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:28:42.768840 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:42.768750    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:42.768840 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:42.768812    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:42.781217 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:42.781185    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:43.256606 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:43.256574    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:28:47.986832 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:28:47.986797    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:29:11.363507 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:11.363470    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"]
Aug 03 16:29:13.093655 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:13.093618    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:29:18.212421 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:18.212392    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:29:18.215342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:18.215318    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:29:24.399693 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:24.399657    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:29:26.382846 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.382791    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h" podUID="92f222c2-30a0-426c-aab9-861f68ee8000" containerName="main" containerID="cri-o://1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24" gracePeriod=45
Aug 03 16:29:26.626043 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.626020    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:29:26.774280 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774189    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-model-cache\") pod \"92f222c2-30a0-426c-aab9-861f68ee8000\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") "
Aug 03 16:29:26.774280 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774256    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-tmp-dir\") pod \"92f222c2-30a0-426c-aab9-861f68ee8000\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") "
Aug 03 16:29:26.774280 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774282    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f222c2-30a0-426c-aab9-861f68ee8000-tls-certs\") pod \"92f222c2-30a0-426c-aab9-861f68ee8000\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") "
Aug 03 16:29:26.774553 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774308    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-dshm\") pod \"92f222c2-30a0-426c-aab9-861f68ee8000\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") "
Aug 03 16:29:26.774553 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wf8s\" (UniqueName: \"kubernetes.io/projected/92f222c2-30a0-426c-aab9-861f68ee8000-kube-api-access-8wf8s\") pod \"92f222c2-30a0-426c-aab9-861f68ee8000\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") "
Aug 03 16:29:26.774553 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774370    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-home\") pod \"92f222c2-30a0-426c-aab9-861f68ee8000\" (UID: \"92f222c2-30a0-426c-aab9-861f68ee8000\") "
Aug 03 16:29:26.774690 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774542    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-model-cache" (OuterVolumeSpecName: "model-cache") pod "92f222c2-30a0-426c-aab9-861f68ee8000" (UID: "92f222c2-30a0-426c-aab9-861f68ee8000"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:29:26.774690 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774639    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92f222c2-30a0-426c-aab9-861f68ee8000" (UID: "92f222c2-30a0-426c-aab9-861f68ee8000"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:29:26.774881 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774691    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-home" (OuterVolumeSpecName: "home") pod "92f222c2-30a0-426c-aab9-861f68ee8000" (UID: "92f222c2-30a0-426c-aab9-861f68ee8000"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:29:26.774881 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774795    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:29:26.774881 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.774812    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:29:26.776671 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.776640    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92f222c2-30a0-426c-aab9-861f68ee8000-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92f222c2-30a0-426c-aab9-861f68ee8000" (UID: "92f222c2-30a0-426c-aab9-861f68ee8000"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:29:26.776800 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.776681    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-dshm" (OuterVolumeSpecName: "dshm") pod "92f222c2-30a0-426c-aab9-861f68ee8000" (UID: "92f222c2-30a0-426c-aab9-861f68ee8000"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:29:26.776800 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.776692    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92f222c2-30a0-426c-aab9-861f68ee8000-kube-api-access-8wf8s" (OuterVolumeSpecName: "kube-api-access-8wf8s") pod "92f222c2-30a0-426c-aab9-861f68ee8000" (UID: "92f222c2-30a0-426c-aab9-861f68ee8000"). InnerVolumeSpecName "kube-api-access-8wf8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:29:26.876307 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.876256    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wf8s\" (UniqueName: \"kubernetes.io/projected/92f222c2-30a0-426c-aab9-861f68ee8000-kube-api-access-8wf8s\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:29:26.876307 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.876302    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:29:26.876307 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.876313    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f222c2-30a0-426c-aab9-861f68ee8000-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:29:26.876307 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:26.876321    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f222c2-30a0-426c-aab9-861f68ee8000-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:29:27.428286 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.428254    2564 generic.go:356] "Generic (PLEG): container finished" podID="92f222c2-30a0-426c-aab9-861f68ee8000" containerID="1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24" exitCode=0
Aug 03 16:29:27.428703 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.428302    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"
Aug 03 16:29:27.428703 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.428311    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h" event={"ID":"92f222c2-30a0-426c-aab9-861f68ee8000","Type":"ContainerDied","Data":"1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24"}
Aug 03 16:29:27.428703 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.428346    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h" event={"ID":"92f222c2-30a0-426c-aab9-861f68ee8000","Type":"ContainerDied","Data":"dc1f7566f4f3688a7a91c51a4e2595cbb6673b47f688e0fb532595582e56df67"}
Aug 03 16:29:27.428703 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.428364    2564 scope.go:117] "RemoveContainer" containerID="1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24"
Aug 03 16:29:27.436842 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.436825    2564 scope.go:117] "RemoveContainer" containerID="1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24"
Aug 03 16:29:27.437091 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:29:27.437072    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24\": container with ID starting with 1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24 not found: ID does not exist" containerID="1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24"
Aug 03 16:29:27.437155 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.437103    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24"} err="failed to get container status \"1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24\": rpc error: code = NotFound desc = could not find container \"1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24\": container with ID starting with 1c9bf90453a852dae6e89fab2314019b560e9712b42f6cf2cd64926f9e036e24 not found: ID does not exist"
Aug 03 16:29:27.451088 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.451054    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"]
Aug 03 16:29:27.456680 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:27.456654    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ed4bc549-kserve-77b7dfdccf-brc6h"]
Aug 03 16:29:28.244864 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:28.244832    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92f222c2-30a0-426c-aab9-861f68ee8000" path="/var/lib/kubelet/pods/92f222c2-30a0-426c-aab9-861f68ee8000/volumes"
Aug 03 16:29:28.499666 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:28.499592    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:29:34.691576 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:34.691542    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:29:45.093085 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:29:45.093045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:30:02.686534 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:30:02.686496    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:30:08.990207 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:30:08.990173    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:30:19.197043 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:30:19.197004    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:30:26.486378 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:30:26.486340    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:30:44.697636 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:30:44.697600    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:30:50.498535 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:30:50.498502    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:30:58.704995 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:30:58.704952    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:31:10.098242 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:31:10.098148    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:31:19.288060 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:31:19.288023    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:31:29.092671 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:31:29.092630    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:31:37.801099 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:31:37.801063    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:31:45.099817 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:31:45.095633    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:31:54.690486 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:31:54.690457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:32:02.098676 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:32:02.098634    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:32:11.903765 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:32:11.903719    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:32:26.986786 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:32:26.986751    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:32:32.891643 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:32:32.891603    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:32:39.197008 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:32:39.196973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:32:50.895948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:32:50.895862    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:32:59.493277 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:32:59.493248    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:33:04.884544 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:33:04.884509    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:33:19.200941 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:33:19.200907    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:33:49.488085 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:33:49.488049    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:33:58.990463 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:33:58.990426    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:34:18.235912 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:34:18.235878    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:34:18.240830 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:34:18.240806    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:34:24.585822 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:34:24.585790    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:34:33.494250 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:34:33.494209    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:34:59.530941 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:34:59.530908    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:35:05.297837 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:35:05.297801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:35:35.898127 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:35:35.898096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:35:48.990321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:35:48.990241    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:37:19.197224 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:37:19.197149    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:37:27.996090 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:37:27.996056    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:37:39.207954 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:37:39.207910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:38:20.788853 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:38:20.788819    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:39:01.417728 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:39:01.417650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:39:18.259994 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:39:18.259966    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:39:18.265996 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:39:18.265973    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:40:46.091459 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:40:46.091422    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:40:53.507368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:40:53.507332    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:41:28.962808 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.962769    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"]
Aug 03 16:41:28.965289 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.963234    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f222c2-30a0-426c-aab9-861f68ee8000" containerName="main"
Aug 03 16:41:28.965289 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.963253    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f222c2-30a0-426c-aab9-861f68ee8000" containerName="main"
Aug 03 16:41:28.965289 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.963320    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f222c2-30a0-426c-aab9-861f68ee8000" containerName="main"
Aug 03 16:41:28.966154 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.966134    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:28.969891 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.969871    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-65805bcc\"/\"default-dockercfg-h64kg\""
Aug 03 16:41:28.969999 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.969871    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-65805bcc\"/\"e2e-lifecycle-model-65805bcc-kserve-self-signed-certs\""
Aug 03 16:41:28.969999 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.969875    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-65805bcc\"/\"openshift-service-ca.crt\""
Aug 03 16:41:28.970171 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.970158    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-65805bcc\"/\"kube-root-ca.crt\""
Aug 03 16:41:28.977020 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:28.976997    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"]
Aug 03 16:41:29.136304 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.136273    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tls-certs\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.136490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.136309    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x689g\" (UniqueName: \"kubernetes.io/projected/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-kube-api-access-x689g\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.136490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.136349    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-dshm\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.136490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.136430    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tmp-dir\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.136589 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.136514    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-model-cache\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.136589 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.136553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-home\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.237590 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-model-cache\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.237590 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-home\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.237822 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tls-certs\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.237822 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x689g\" (UniqueName: \"kubernetes.io/projected/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-kube-api-access-x689g\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.237822 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-dshm\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.237822 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tmp-dir\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.238008 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237943    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-model-cache\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.238008 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.237974    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-home\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.238099 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.238070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tmp-dir\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.239854 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.239830    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-dshm\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.240180 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.240162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tls-certs\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.246966 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.246945    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x689g\" (UniqueName: \"kubernetes.io/projected/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-kube-api-access-x689g\") pod \"e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") " pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.274745 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.274721    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:29.396650 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.396624    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"]
Aug 03 16:41:29.398996 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:41:29.398967    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f732d99_a589_4f10_a1a3_4fea5d9ccb59.slice/crio-e03db5e83586a1b94302fafd85d820f7ad21dbd4bc14ecec3e2cb858fec1ba3a WatchSource:0}: Error finding container e03db5e83586a1b94302fafd85d820f7ad21dbd4bc14ecec3e2cb858fec1ba3a: Status 404 returned error can't find the container with id e03db5e83586a1b94302fafd85d820f7ad21dbd4bc14ecec3e2cb858fec1ba3a
Aug 03 16:41:29.430675 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:29.430638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4" event={"ID":"2f732d99-a589-4f10-a1a3-4fea5d9ccb59","Type":"ContainerStarted","Data":"e03db5e83586a1b94302fafd85d820f7ad21dbd4bc14ecec3e2cb858fec1ba3a"}
Aug 03 16:41:30.436656 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:30.436617    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4" event={"ID":"2f732d99-a589-4f10-a1a3-4fea5d9ccb59","Type":"ContainerStarted","Data":"bf8325ff0f53de7a20547ba5f32d4bafc46bc387289bc0290cf22fc86cfb5ab7"}
Aug 03 16:41:30.455078 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:30.455029    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4" podStartSLOduration=2.455015838 podStartE2EDuration="2.455015838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:41:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:41:30.453887416 +0000 UTC m=+2232.720703424" watchObservedRunningTime="2026-08-03 16:41:30.455015838 +0000 UTC m=+2232.721831844"
Aug 03 16:41:31.191290 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:31.191252    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:41:39.275570 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:39.275535    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:39.275570 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:39.275577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:39.288005 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:39.287980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:39.491976 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:39.491951    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:41:50.003831 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:41:50.003795    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:42:13.493115 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:13.493080    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:42:19.088874 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:19.088832    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:42:24.576725 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:24.576687    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"]
Aug 03 16:42:39.595810 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.595766    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4" podUID="2f732d99-a589-4f10-a1a3-4fea5d9ccb59" containerName="main" containerID="cri-o://bf8325ff0f53de7a20547ba5f32d4bafc46bc387289bc0290cf22fc86cfb5ab7" gracePeriod=45
Aug 03 16:42:39.726457 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.726425    2564 generic.go:356] "Generic (PLEG): container finished" podID="2f732d99-a589-4f10-a1a3-4fea5d9ccb59" containerID="bf8325ff0f53de7a20547ba5f32d4bafc46bc387289bc0290cf22fc86cfb5ab7" exitCode=0
Aug 03 16:42:39.726589 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.726500    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4" event={"ID":"2f732d99-a589-4f10-a1a3-4fea5d9ccb59","Type":"ContainerDied","Data":"bf8325ff0f53de7a20547ba5f32d4bafc46bc387289bc0290cf22fc86cfb5ab7"}
Aug 03 16:42:39.844007 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.843982    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:42:39.937285 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tmp-dir\") pod \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") "
Aug 03 16:42:39.937285 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937259    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-model-cache\") pod \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") "
Aug 03 16:42:39.937285 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937285    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-dshm\") pod \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") "
Aug 03 16:42:39.937569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937310    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tls-certs\") pod \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") "
Aug 03 16:42:39.937569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937333    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x689g\" (UniqueName: \"kubernetes.io/projected/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-kube-api-access-x689g\") pod \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") "
Aug 03 16:42:39.937569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937404    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-home\") pod \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\" (UID: \"2f732d99-a589-4f10-a1a3-4fea5d9ccb59\") "
Aug 03 16:42:39.937569 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937436    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2f732d99-a589-4f10-a1a3-4fea5d9ccb59" (UID: "2f732d99-a589-4f10-a1a3-4fea5d9ccb59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:42:39.937766 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937592    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-model-cache" (OuterVolumeSpecName: "model-cache") pod "2f732d99-a589-4f10-a1a3-4fea5d9ccb59" (UID: "2f732d99-a589-4f10-a1a3-4fea5d9ccb59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:42:39.937766 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937665    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:42:39.937766 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937684    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:42:39.937766 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.937690    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-home" (OuterVolumeSpecName: "home") pod "2f732d99-a589-4f10-a1a3-4fea5d9ccb59" (UID: "2f732d99-a589-4f10-a1a3-4fea5d9ccb59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:42:39.939551 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.939526    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f732d99-a589-4f10-a1a3-4fea5d9ccb59" (UID: "2f732d99-a589-4f10-a1a3-4fea5d9ccb59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:42:39.939658 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.939574    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-dshm" (OuterVolumeSpecName: "dshm") pod "2f732d99-a589-4f10-a1a3-4fea5d9ccb59" (UID: "2f732d99-a589-4f10-a1a3-4fea5d9ccb59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:42:39.939658 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:39.939591    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-kube-api-access-x689g" (OuterVolumeSpecName: "kube-api-access-x689g") pod "2f732d99-a589-4f10-a1a3-4fea5d9ccb59" (UID: "2f732d99-a589-4f10-a1a3-4fea5d9ccb59"). InnerVolumeSpecName "kube-api-access-x689g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:42:40.038379 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.038346    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:42:40.038379 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.038371    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:42:40.038379 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.038382    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x689g\" (UniqueName: \"kubernetes.io/projected/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-kube-api-access-x689g\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:42:40.038603 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.038391    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f732d99-a589-4f10-a1a3-4fea5d9ccb59-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:42:40.733053 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.733021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4" event={"ID":"2f732d99-a589-4f10-a1a3-4fea5d9ccb59","Type":"ContainerDied","Data":"e03db5e83586a1b94302fafd85d820f7ad21dbd4bc14ecec3e2cb858fec1ba3a"}
Aug 03 16:42:40.733053 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.733041    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"
Aug 03 16:42:40.733552 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.733070    2564 scope.go:117] "RemoveContainer" containerID="bf8325ff0f53de7a20547ba5f32d4bafc46bc387289bc0290cf22fc86cfb5ab7"
Aug 03 16:42:40.751481 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.751458    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"]
Aug 03 16:42:40.758085 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:40.758062    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-65805bcc/e2e-lifecycle-model-65805bcc-kserve-66f4cfcc9-kj2p4"]
Aug 03 16:42:42.243788 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:42:42.243753    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f732d99-a589-4f10-a1a3-4fea5d9ccb59" path="/var/lib/kubelet/pods/2f732d99-a589-4f10-a1a3-4fea5d9ccb59/volumes"
Aug 03 16:43:16.093475 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:43:16.093375    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:43:28.690591 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:43:28.690554    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:43:35.201064 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:43:35.201030    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:43:43.894242 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:43:43.894203    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:44:18.282655 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:44:18.282628    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:44:18.290335 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:44:18.290315    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:44:27.490192 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:44:27.490153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:44:55.788428 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:44:55.788388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:45:00.136673 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.136633    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp"]
Aug 03 16:45:00.137081 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.136973    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f732d99-a589-4f10-a1a3-4fea5d9ccb59" containerName="main"
Aug 03 16:45:00.137081 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.136986    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f732d99-a589-4f10-a1a3-4fea5d9ccb59" containerName="main"
Aug 03 16:45:00.137081 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.137052    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f732d99-a589-4f10-a1a3-4fea5d9ccb59" containerName="main"
Aug 03 16:45:00.139977 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.139957    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"]
Aug 03 16:45:00.140094 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.140073    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp"
Aug 03 16:45:00.142645 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.142625    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:45:00.142960 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.142945    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"
Aug 03 16:45:00.151077 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.150944    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"]
Aug 03 16:45:00.152510 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.152489    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp"
Aug 03 16:45:00.153568 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.153548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp"]
Aug 03 16:45:00.162124 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.162104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"
Aug 03 16:45:00.296282 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.296241    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp"]
Aug 03 16:45:00.299351 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:45:00.299314    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb963e35b_db24_498f_8aff_d11668886f25.slice/crio-4f9a002d19f9741f72d02c1e1d1505b00fea7ae2fe8330da0e00e919ce26e3f7 WatchSource:0}: Error finding container 4f9a002d19f9741f72d02c1e1d1505b00fea7ae2fe8330da0e00e919ce26e3f7: Status 404 returned error can't find the container with id 4f9a002d19f9741f72d02c1e1d1505b00fea7ae2fe8330da0e00e919ce26e3f7
Aug 03 16:45:00.302150 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.302126    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:45:00.322186 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.322156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp" event={"ID":"b963e35b-db24-498f-8aff-d11668886f25","Type":"ContainerStarted","Data":"4f9a002d19f9741f72d02c1e1d1505b00fea7ae2fe8330da0e00e919ce26e3f7"}
Aug 03 16:45:00.327412 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:00.327391    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"]
Aug 03 16:45:00.329533 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:45:00.329501    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64880d5_76c0_4d44_a5b5_0a2384a4efe3.slice/crio-d480a6206b7b6a759471264fb3e4fa638f1fb257a1bcf9c864493b87f3ee27f3 WatchSource:0}: Error finding container d480a6206b7b6a759471264fb3e4fa638f1fb257a1bcf9c864493b87f3ee27f3: Status 404 returned error can't find the container with id d480a6206b7b6a759471264fb3e4fa638f1fb257a1bcf9c864493b87f3ee27f3
Aug 03 16:45:01.327220 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:01.327186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh" event={"ID":"f64880d5-76c0-4d44-a5b5-0a2384a4efe3","Type":"ContainerStarted","Data":"d480a6206b7b6a759471264fb3e4fa638f1fb257a1bcf9c864493b87f3ee27f3"}
Aug 03 16:45:03.338295 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:03.338258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh" event={"ID":"f64880d5-76c0-4d44-a5b5-0a2384a4efe3","Type":"ContainerStarted","Data":"b72745f24908f3f2ad1f8a97fa9c568da562a75a824207f5c99bb1cd55a17c15"}
Aug 03 16:45:03.339696 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:03.339672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp" event={"ID":"b963e35b-db24-498f-8aff-d11668886f25","Type":"ContainerStarted","Data":"5bbfaeb711760cc94f4d6bdd5645390b744c4918b9bb2f2276fd25e94810b3c7"}
Aug 03 16:45:03.356207 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:03.356161    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh" podStartSLOduration=0.776243654 podStartE2EDuration="3.356146514s" totalImagesPullingTime="2.57990286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:45:03.354590094 +0000 UTC m=+2445.621406102" watchObservedRunningTime="2026-08-03 16:45:03.356146514 +0000 UTC m=+2445.622962520"
Aug 03 16:45:03.371012 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:03.370968    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp" podStartSLOduration=0.757702166 podStartE2EDuration="3.37095505s" totalImagesPullingTime="2.613252884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:45:03.370468186 +0000 UTC m=+2445.637284197" watchObservedRunningTime="2026-08-03 16:45:03.37095505 +0000 UTC m=+2445.637771056"
Aug 03 16:45:04.345936 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:04.345904    2564 generic.go:356] "Generic (PLEG): container finished" podID="f64880d5-76c0-4d44-a5b5-0a2384a4efe3" containerID="b72745f24908f3f2ad1f8a97fa9c568da562a75a824207f5c99bb1cd55a17c15" exitCode=0
Aug 03 16:45:04.346400 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:04.345997    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh" event={"ID":"f64880d5-76c0-4d44-a5b5-0a2384a4efe3","Type":"ContainerDied","Data":"b72745f24908f3f2ad1f8a97fa9c568da562a75a824207f5c99bb1cd55a17c15"}
Aug 03 16:45:04.347495 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:04.347474    2564 generic.go:356] "Generic (PLEG): container finished" podID="b963e35b-db24-498f-8aff-d11668886f25" containerID="5bbfaeb711760cc94f4d6bdd5645390b744c4918b9bb2f2276fd25e94810b3c7" exitCode=0
Aug 03 16:45:04.347575 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:04.347507    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp" event={"ID":"b963e35b-db24-498f-8aff-d11668886f25","Type":"ContainerDied","Data":"5bbfaeb711760cc94f4d6bdd5645390b744c4918b9bb2f2276fd25e94810b3c7"}
Aug 03 16:45:05.508078 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:05.508057    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp"
Aug 03 16:45:05.510837 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:05.510820    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"
Aug 03 16:45:06.358490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:06.358462    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp"
Aug 03 16:45:06.358666 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:06.358464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762925-v8qwp" event={"ID":"b963e35b-db24-498f-8aff-d11668886f25","Type":"ContainerDied","Data":"4f9a002d19f9741f72d02c1e1d1505b00fea7ae2fe8330da0e00e919ce26e3f7"}
Aug 03 16:45:06.358666 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:06.358596    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f9a002d19f9741f72d02c1e1d1505b00fea7ae2fe8330da0e00e919ce26e3f7"
Aug 03 16:45:06.359794 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:06.359772    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"
Aug 03 16:45:06.359919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:06.359771    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh" event={"ID":"f64880d5-76c0-4d44-a5b5-0a2384a4efe3","Type":"ContainerDied","Data":"d480a6206b7b6a759471264fb3e4fa638f1fb257a1bcf9c864493b87f3ee27f3"}
Aug 03 16:45:06.359919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:06.359893    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d480a6206b7b6a759471264fb3e4fa638f1fb257a1bcf9c864493b87f3ee27f3"
Aug 03 16:45:50.957443 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.957407    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"]
Aug 03 16:45:50.958059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.957913    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64880d5-76c0-4d44-a5b5-0a2384a4efe3" containerName="cleanup"
Aug 03 16:45:50.958059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.957932    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64880d5-76c0-4d44-a5b5-0a2384a4efe3" containerName="cleanup"
Aug 03 16:45:50.958059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.957945    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b963e35b-db24-498f-8aff-d11668886f25" containerName="cleanup"
Aug 03 16:45:50.958059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.957954    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b963e35b-db24-498f-8aff-d11668886f25" containerName="cleanup"
Aug 03 16:45:50.958059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.958049    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b963e35b-db24-498f-8aff-d11668886f25" containerName="cleanup"
Aug 03 16:45:50.958059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.958061    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64880d5-76c0-4d44-a5b5-0a2384a4efe3" containerName="cleanup"
Aug 03 16:45:50.963899 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.963880    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:50.966865 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.966841    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7ca182\"/\"default-dockercfg-cvjjs\""
Aug 03 16:45:50.967923 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.967905    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7ca182\"/\"kube-root-ca.crt\""
Aug 03 16:45:50.968045 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.967935    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7ca182\"/\"openshift-service-ca.crt\""
Aug 03 16:45:50.968045 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.967941    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7ca182\"/\"test-model-7ca182-kserve-self-signed-certs\""
Aug 03 16:45:50.971761 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.971725    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"]
Aug 03 16:45:50.997188 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.997162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-dshm\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:50.997188 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.997188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-model-cache\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:50.997336 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.997268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wbt\" (UniqueName: \"kubernetes.io/projected/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-kube-api-access-79wbt\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:50.997336 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.997310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-home\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:50.997336 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.997332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tmp-dir\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:50.997433 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:50.997390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tls-certs\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098014 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.097969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-dshm\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098014 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-model-cache\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098239 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79wbt\" (UniqueName: \"kubernetes.io/projected/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-kube-api-access-79wbt\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098239 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-home\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098239 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tmp-dir\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098239 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098126    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tls-certs\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098499 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098479    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-home\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098557 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-model-cache\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.098609 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.098549    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tmp-dir\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.100193 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.100173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-dshm\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.100431 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.100415    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tls-certs\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.108225 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.108200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wbt\" (UniqueName: \"kubernetes.io/projected/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-kube-api-access-79wbt\") pod \"test-model-7ca182-kserve-b59d6df8f-mflz4\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") " pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.273661 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.273563    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:45:51.405128 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.405104    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"]
Aug 03 16:45:51.406801 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:45:51.406773    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16b5b97e_cd48_4bc3_a660_2879eaf52a6c.slice/crio-19ab8665b4ca250977cbc4aea8fc3730a7ab5614a05edecbb43636f75fd50e9b WatchSource:0}: Error finding container 19ab8665b4ca250977cbc4aea8fc3730a7ab5614a05edecbb43636f75fd50e9b: Status 404 returned error can't find the container with id 19ab8665b4ca250977cbc4aea8fc3730a7ab5614a05edecbb43636f75fd50e9b
Aug 03 16:45:51.561711 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.561684    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4" event={"ID":"16b5b97e-cd48-4bc3-a660-2879eaf52a6c","Type":"ContainerStarted","Data":"10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9"}
Aug 03 16:45:51.561882 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.561730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4" event={"ID":"16b5b97e-cd48-4bc3-a660-2879eaf52a6c","Type":"ContainerStarted","Data":"19ab8665b4ca250977cbc4aea8fc3730a7ab5614a05edecbb43636f75fd50e9b"}
Aug 03 16:45:51.583016 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:51.582968    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4" podStartSLOduration=1.582954132 podStartE2EDuration="1.582954132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:45:51.582391801 +0000 UTC m=+2493.849207809" watchObservedRunningTime="2026-08-03 16:45:51.582954132 +0000 UTC m=+2493.849770138"
Aug 03 16:45:53.193498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:45:53.193454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:46:01.274260 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:01.274220    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:46:01.274802 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:01.274279    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:46:01.287349 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:01.287326    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:46:01.411255 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:01.411225    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:46:01.621369 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:01.621341    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:46:09.459693 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.459651    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"]
Aug 03 16:46:09.462975 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.462952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.466628 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.466602    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d7cee2\"/\"default-dockercfg-l52k5\""
Aug 03 16:46:09.466628 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.466612    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d7cee2\"/\"openshift-service-ca.crt\""
Aug 03 16:46:09.466833 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.466659    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d7cee2\"/\"kube-root-ca.crt\""
Aug 03 16:46:09.466833 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.466612    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d7cee2\"/\"test-model-d7cee2-kserve-self-signed-certs\""
Aug 03 16:46:09.472126 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.472096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"]
Aug 03 16:46:09.566093 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.566063    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c109c363-5e22-430d-8930-52c2a9442c19-tls-certs\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.566262 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.566106    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smr4d\" (UniqueName: \"kubernetes.io/projected/c109c363-5e22-430d-8930-52c2a9442c19-kube-api-access-smr4d\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.566262 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.566159    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-home\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.566340 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.566263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-dshm\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.566340 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.566293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-model-cache\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.566340 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.566315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-tmp-dir\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.667671 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.667643    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-model-cache\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.667671 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.667675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-tmp-dir\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.667920 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.667718    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c109c363-5e22-430d-8930-52c2a9442c19-tls-certs\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.667920 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.667767    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smr4d\" (UniqueName: \"kubernetes.io/projected/c109c363-5e22-430d-8930-52c2a9442c19-kube-api-access-smr4d\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.667920 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.667788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-home\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.667920 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.667845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-dshm\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.668180 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.668155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-model-cache\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.668249 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.668155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-tmp-dir\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.668249 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.668222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-home\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.670096 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.670074    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-dshm\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.670352 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.670336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c109c363-5e22-430d-8930-52c2a9442c19-tls-certs\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.675888 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.675866    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smr4d\" (UniqueName: \"kubernetes.io/projected/c109c363-5e22-430d-8930-52c2a9442c19-kube-api-access-smr4d\") pod \"test-model-d7cee2-kserve-6cd659d785-pz69l\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") " pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.772035 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.771970    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:09.900956 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:09.900922    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"]
Aug 03 16:46:09.903544 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:46:09.903511    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc109c363_5e22_430d_8930_52c2a9442c19.slice/crio-3f9fa683943ca17c3c444486cca99cb1060c73b8470eda7b3d7297257b58f02f WatchSource:0}: Error finding container 3f9fa683943ca17c3c444486cca99cb1060c73b8470eda7b3d7297257b58f02f: Status 404 returned error can't find the container with id 3f9fa683943ca17c3c444486cca99cb1060c73b8470eda7b3d7297257b58f02f
Aug 03 16:46:10.655911 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:10.655870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l" event={"ID":"c109c363-5e22-430d-8930-52c2a9442c19","Type":"ContainerStarted","Data":"926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084"}
Aug 03 16:46:10.655911 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:10.655913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l" event={"ID":"c109c363-5e22-430d-8930-52c2a9442c19","Type":"ContainerStarted","Data":"3f9fa683943ca17c3c444486cca99cb1060c73b8470eda7b3d7297257b58f02f"}
Aug 03 16:46:10.673727 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:10.673676    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l" podStartSLOduration=1.673659045 podStartE2EDuration="1.673659045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:46:10.672224057 +0000 UTC m=+2512.939040065" watchObservedRunningTime="2026-08-03 16:46:10.673659045 +0000 UTC m=+2512.940475051"
Aug 03 16:46:13.296722 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:13.296692    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:46:19.772451 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:19.772414    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:19.772874 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:19.772465    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:19.785118 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:19.785094    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:20.708730 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:20.708703    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:46:52.532892 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:52.532858    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"]
Aug 03 16:46:53.426287 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:53.426257    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"]
Aug 03 16:46:55.796498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:46:55.796458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:47:06.195263 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:06.195228    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:47:07.551795 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.551752    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4" podUID="16b5b97e-cd48-4bc3-a660-2879eaf52a6c" containerName="main" containerID="cri-o://10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9" gracePeriod=45
Aug 03 16:47:07.801551 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.801520    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:47:07.881915 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.881884    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tmp-dir\") pod \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") "
Aug 03 16:47:07.882084 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.881940    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-home\") pod \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") "
Aug 03 16:47:07.882084 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.881972    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79wbt\" (UniqueName: \"kubernetes.io/projected/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-kube-api-access-79wbt\") pod \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") "
Aug 03 16:47:07.882084 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882001    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-dshm\") pod \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") "
Aug 03 16:47:07.882084 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882030    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tls-certs\") pod \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") "
Aug 03 16:47:07.882084 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882056    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-model-cache\") pod \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\" (UID: \"16b5b97e-cd48-4bc3-a660-2879eaf52a6c\") "
Aug 03 16:47:07.882334 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882207    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16b5b97e-cd48-4bc3-a660-2879eaf52a6c" (UID: "16b5b97e-cd48-4bc3-a660-2879eaf52a6c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:07.882334 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882230    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-home" (OuterVolumeSpecName: "home") pod "16b5b97e-cd48-4bc3-a660-2879eaf52a6c" (UID: "16b5b97e-cd48-4bc3-a660-2879eaf52a6c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:07.882439 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882354    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:07.882439 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882373    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:07.882439 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.882422    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-model-cache" (OuterVolumeSpecName: "model-cache") pod "16b5b97e-cd48-4bc3-a660-2879eaf52a6c" (UID: "16b5b97e-cd48-4bc3-a660-2879eaf52a6c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:07.884342 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.884310    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-kube-api-access-79wbt" (OuterVolumeSpecName: "kube-api-access-79wbt") pod "16b5b97e-cd48-4bc3-a660-2879eaf52a6c" (UID: "16b5b97e-cd48-4bc3-a660-2879eaf52a6c"). InnerVolumeSpecName "kube-api-access-79wbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:07.884444 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.884338    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16b5b97e-cd48-4bc3-a660-2879eaf52a6c" (UID: "16b5b97e-cd48-4bc3-a660-2879eaf52a6c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:47:07.884444 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.884351    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-dshm" (OuterVolumeSpecName: "dshm") pod "16b5b97e-cd48-4bc3-a660-2879eaf52a6c" (UID: "16b5b97e-cd48-4bc3-a660-2879eaf52a6c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:07.905432 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.905408    2564 generic.go:356] "Generic (PLEG): container finished" podID="16b5b97e-cd48-4bc3-a660-2879eaf52a6c" containerID="10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9" exitCode=0
Aug 03 16:47:07.905558 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.905448    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"
Aug 03 16:47:07.905558 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.905464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4" event={"ID":"16b5b97e-cd48-4bc3-a660-2879eaf52a6c","Type":"ContainerDied","Data":"10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9"}
Aug 03 16:47:07.905558 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.905511    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4" event={"ID":"16b5b97e-cd48-4bc3-a660-2879eaf52a6c","Type":"ContainerDied","Data":"19ab8665b4ca250977cbc4aea8fc3730a7ab5614a05edecbb43636f75fd50e9b"}
Aug 03 16:47:07.905558 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.905528    2564 scope.go:117] "RemoveContainer" containerID="10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9"
Aug 03 16:47:07.914359 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.914345    2564 scope.go:117] "RemoveContainer" containerID="10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9"
Aug 03 16:47:07.914596 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:47:07.914580    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9\": container with ID starting with 10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9 not found: ID does not exist" containerID="10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9"
Aug 03 16:47:07.914648 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.914605    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9"} err="failed to get container status \"10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9\": rpc error: code = NotFound desc = could not find container \"10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9\": container with ID starting with 10ac44590753c18432b76f603e05491d628997dabc3cf1452fd0887f8b4b74c9 not found: ID does not exist"
Aug 03 16:47:07.938475 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.938443    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"]
Aug 03 16:47:07.947266 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.947241    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7ca182/test-model-7ca182-kserve-b59d6df8f-mflz4"]
Aug 03 16:47:07.983028 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.982998    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79wbt\" (UniqueName: \"kubernetes.io/projected/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-kube-api-access-79wbt\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:07.983028 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.983024    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:07.983173 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.983036    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:07.983173 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:07.983045    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b5b97e-cd48-4bc3-a660-2879eaf52a6c-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:08.242970 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.242891    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16b5b97e-cd48-4bc3-a660-2879eaf52a6c" path="/var/lib/kubelet/pods/16b5b97e-cd48-4bc3-a660-2879eaf52a6c/volumes"
Aug 03 16:47:08.446177 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.446119    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l" podUID="c109c363-5e22-430d-8930-52c2a9442c19" containerName="main" containerID="cri-o://926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084" gracePeriod=45
Aug 03 16:47:08.689016 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.688992    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:47:08.789814 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.789777    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-model-cache\") pod \"c109c363-5e22-430d-8930-52c2a9442c19\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") "
Aug 03 16:47:08.790025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.789841    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c109c363-5e22-430d-8930-52c2a9442c19-tls-certs\") pod \"c109c363-5e22-430d-8930-52c2a9442c19\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") "
Aug 03 16:47:08.790025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.789887    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smr4d\" (UniqueName: \"kubernetes.io/projected/c109c363-5e22-430d-8930-52c2a9442c19-kube-api-access-smr4d\") pod \"c109c363-5e22-430d-8930-52c2a9442c19\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") "
Aug 03 16:47:08.790025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.789905    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-tmp-dir\") pod \"c109c363-5e22-430d-8930-52c2a9442c19\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") "
Aug 03 16:47:08.790025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.789940    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-dshm\") pod \"c109c363-5e22-430d-8930-52c2a9442c19\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") "
Aug 03 16:47:08.790025 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.789982    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-home\") pod \"c109c363-5e22-430d-8930-52c2a9442c19\" (UID: \"c109c363-5e22-430d-8930-52c2a9442c19\") "
Aug 03 16:47:08.790282 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.790121    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-model-cache" (OuterVolumeSpecName: "model-cache") pod "c109c363-5e22-430d-8930-52c2a9442c19" (UID: "c109c363-5e22-430d-8930-52c2a9442c19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:08.790282 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.790211    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:08.790282 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.790249    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c109c363-5e22-430d-8930-52c2a9442c19" (UID: "c109c363-5e22-430d-8930-52c2a9442c19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:08.790427 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.790285    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-home" (OuterVolumeSpecName: "home") pod "c109c363-5e22-430d-8930-52c2a9442c19" (UID: "c109c363-5e22-430d-8930-52c2a9442c19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:08.792223 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.792202    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c109c363-5e22-430d-8930-52c2a9442c19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c109c363-5e22-430d-8930-52c2a9442c19" (UID: "c109c363-5e22-430d-8930-52c2a9442c19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:47:08.792338 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.792240    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-dshm" (OuterVolumeSpecName: "dshm") pod "c109c363-5e22-430d-8930-52c2a9442c19" (UID: "c109c363-5e22-430d-8930-52c2a9442c19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:47:08.792338 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.792255    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c109c363-5e22-430d-8930-52c2a9442c19-kube-api-access-smr4d" (OuterVolumeSpecName: "kube-api-access-smr4d") pod "c109c363-5e22-430d-8930-52c2a9442c19" (UID: "c109c363-5e22-430d-8930-52c2a9442c19"). InnerVolumeSpecName "kube-api-access-smr4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:08.890875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.890803    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c109c363-5e22-430d-8930-52c2a9442c19-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:08.890875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.890836    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smr4d\" (UniqueName: \"kubernetes.io/projected/c109c363-5e22-430d-8930-52c2a9442c19-kube-api-access-smr4d\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:08.890875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.890847    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:08.890875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.890855    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:08.890875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.890862    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c109c363-5e22-430d-8930-52c2a9442c19-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:47:08.910365 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.910341    2564 generic.go:356] "Generic (PLEG): container finished" podID="c109c363-5e22-430d-8930-52c2a9442c19" containerID="926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084" exitCode=0
Aug 03 16:47:08.910493 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.910385    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"
Aug 03 16:47:08.910493 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.910424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l" event={"ID":"c109c363-5e22-430d-8930-52c2a9442c19","Type":"ContainerDied","Data":"926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084"}
Aug 03 16:47:08.910493 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.910456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l" event={"ID":"c109c363-5e22-430d-8930-52c2a9442c19","Type":"ContainerDied","Data":"3f9fa683943ca17c3c444486cca99cb1060c73b8470eda7b3d7297257b58f02f"}
Aug 03 16:47:08.910493 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.910475    2564 scope.go:117] "RemoveContainer" containerID="926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084"
Aug 03 16:47:08.918496 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.918476    2564 scope.go:117] "RemoveContainer" containerID="926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084"
Aug 03 16:47:08.918772 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:47:08.918748    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084\": container with ID starting with 926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084 not found: ID does not exist" containerID="926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084"
Aug 03 16:47:08.918861 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.918779    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084"} err="failed to get container status \"926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084\": rpc error: code = NotFound desc = could not find container \"926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084\": container with ID starting with 926da6e1877083d43c384c92077938f71b006cbdcfdab697b86841692b5e4084 not found: ID does not exist"
Aug 03 16:47:08.937248 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.937219    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"]
Aug 03 16:47:08.940709 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:08.940687    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d7cee2/test-model-d7cee2-kserve-6cd659d785-pz69l"]
Aug 03 16:47:10.243134 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:10.243103    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c109c363-5e22-430d-8930-52c2a9442c19" path="/var/lib/kubelet/pods/c109c363-5e22-430d-8930-52c2a9442c19/volumes"
Aug 03 16:47:21.916965 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:21.916933    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"]
Aug 03 16:47:21.919971 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:21.919946    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d7cee2-29762925-hggwh"]
Aug 03 16:47:22.243261 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:22.243177    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64880d5-76c0-4d44-a5b5-0a2384a4efe3" path="/var/lib/kubelet/pods/f64880d5-76c0-4d44-a5b5-0a2384a4efe3/volumes"
Aug 03 16:47:23.998005 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:23.997978    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:47:36.784993 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:47:36.784955    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:48:06.091317 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:06.091287    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:48:30.232090 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.232056    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"]
Aug 03 16:48:30.232549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.232354    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b5b97e-cd48-4bc3-a660-2879eaf52a6c" containerName="main"
Aug 03 16:48:30.232549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.232364    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b5b97e-cd48-4bc3-a660-2879eaf52a6c" containerName="main"
Aug 03 16:48:30.232549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.232383    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c109c363-5e22-430d-8930-52c2a9442c19" containerName="main"
Aug 03 16:48:30.232549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.232388    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c109c363-5e22-430d-8930-52c2a9442c19" containerName="main"
Aug 03 16:48:30.232549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.232435    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c109c363-5e22-430d-8930-52c2a9442c19" containerName="main"
Aug 03 16:48:30.232549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.232446    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b5b97e-cd48-4bc3-a660-2879eaf52a6c" containerName="main"
Aug 03 16:48:30.235333 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.235317    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"
Aug 03 16:48:30.237919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.237893    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 03 16:48:30.237919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.237901    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:48:30.244583 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.244519    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"]
Aug 03 16:48:30.396442 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.396409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bee8f33-4c80-48f4-840c-b97513abad84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-deec1a-dxq55\" (UID: \"8bee8f33-4c80-48f4-840c-b97513abad84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"
Aug 03 16:48:30.497441 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.497359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bee8f33-4c80-48f4-840c-b97513abad84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-deec1a-dxq55\" (UID: \"8bee8f33-4c80-48f4-840c-b97513abad84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"
Aug 03 16:48:30.498000 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.497981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bee8f33-4c80-48f4-840c-b97513abad84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-deec1a-dxq55\" (UID: \"8bee8f33-4c80-48f4-840c-b97513abad84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"
Aug 03 16:48:30.546904 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.546874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"
Aug 03 16:48:30.671034 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:30.671010    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"]
Aug 03 16:48:30.672960 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:48:30.672928    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bee8f33_4c80_48f4_840c_b97513abad84.slice/crio-e4712b0d24efff0a7222051a0edfc8b7fb47b37ca912e3c46bc336d0c28c0ea9 WatchSource:0}: Error finding container e4712b0d24efff0a7222051a0edfc8b7fb47b37ca912e3c46bc336d0c28c0ea9: Status 404 returned error can't find the container with id e4712b0d24efff0a7222051a0edfc8b7fb47b37ca912e3c46bc336d0c28c0ea9
Aug 03 16:48:31.262839 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:31.262717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55" event={"ID":"8bee8f33-4c80-48f4-840c-b97513abad84","Type":"ContainerStarted","Data":"089cbd2066c6634b7f9d17c48d7c3955942cf0648291f68d730e7cc0bba899b5"}
Aug 03 16:48:31.262839 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:31.262789    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55" event={"ID":"8bee8f33-4c80-48f4-840c-b97513abad84","Type":"ContainerStarted","Data":"e4712b0d24efff0a7222051a0edfc8b7fb47b37ca912e3c46bc336d0c28c0ea9"}
Aug 03 16:48:31.280357 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:31.280313    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55" podStartSLOduration=1.2802978440000001 podStartE2EDuration="1.280297844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:48:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:48:31.279660811 +0000 UTC m=+2653.546476817" watchObservedRunningTime="2026-08-03 16:48:31.280297844 +0000 UTC m=+2653.547113850"
Aug 03 16:48:32.268919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:32.268886    2564 generic.go:356] "Generic (PLEG): container finished" podID="8bee8f33-4c80-48f4-840c-b97513abad84" containerID="089cbd2066c6634b7f9d17c48d7c3955942cf0648291f68d730e7cc0bba899b5" exitCode=0
Aug 03 16:48:32.269308 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:32.268925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55" event={"ID":"8bee8f33-4c80-48f4-840c-b97513abad84","Type":"ContainerDied","Data":"089cbd2066c6634b7f9d17c48d7c3955942cf0648291f68d730e7cc0bba899b5"}
Aug 03 16:48:33.397368 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:33.397346    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"
Aug 03 16:48:33.530352 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:33.530270    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bee8f33-4c80-48f4-840c-b97513abad84-maas-api-service-ca\") pod \"8bee8f33-4c80-48f4-840c-b97513abad84\" (UID: \"8bee8f33-4c80-48f4-840c-b97513abad84\") "
Aug 03 16:48:33.530612 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:33.530589    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bee8f33-4c80-48f4-840c-b97513abad84-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8bee8f33-4c80-48f4-840c-b97513abad84" (UID: "8bee8f33-4c80-48f4-840c-b97513abad84"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:48:33.630985 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:33.630952    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bee8f33-4c80-48f4-840c-b97513abad84-maas-api-service-ca\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:48:34.278873 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:34.278841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55" event={"ID":"8bee8f33-4c80-48f4-840c-b97513abad84","Type":"ContainerDied","Data":"e4712b0d24efff0a7222051a0edfc8b7fb47b37ca912e3c46bc336d0c28c0ea9"}
Aug 03 16:48:34.279016 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:34.278886    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4712b0d24efff0a7222051a0edfc8b7fb47b37ca912e3c46bc336d0c28c0ea9"
Aug 03 16:48:34.279016 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:34.278851    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"
Aug 03 16:48:41.717895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:41.717859    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"]
Aug 03 16:48:41.727225 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:41.727196    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-deec1a-dxq55"]
Aug 03 16:48:42.248642 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:42.248598    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bee8f33-4c80-48f4-840c-b97513abad84" path="/var/lib/kubelet/pods/8bee8f33-4c80-48f4-840c-b97513abad84/volumes"
Aug 03 16:48:43.591787 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:48:43.591754    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:49:07.294602 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:49:07.294567    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:49:18.307695 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:49:18.307626    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:49:18.315948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:49:18.315923    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:49:20.104605 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:49:20.104569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:49:50.115490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:49:50.115457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:50:16.787802 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.787766    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"]
Aug 03 16:50:16.788273 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.788213    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bee8f33-4c80-48f4-840c-b97513abad84" containerName="revoke-keys"
Aug 03 16:50:16.788273 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.788230    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bee8f33-4c80-48f4-840c-b97513abad84" containerName="revoke-keys"
Aug 03 16:50:16.788383 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.788311    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bee8f33-4c80-48f4-840c-b97513abad84" containerName="revoke-keys"
Aug 03 16:50:16.790650 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.790630    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.799442 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.799423    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"kube-root-ca.crt\""
Aug 03 16:50:16.799442 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.799432    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"openshift-service-ca.crt\""
Aug 03 16:50:16.799549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.799433    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"default-dockercfg-whcnp\""
Aug 03 16:50:16.799549 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.799423    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"auth-test-model-0fae8e-kserve-self-signed-certs\""
Aug 03 16:50:16.811183 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.811157    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-tmp-dir\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.811299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.811195    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd25z\" (UniqueName: \"kubernetes.io/projected/40366669-f476-4195-98b5-cd98f85fd8db-kube-api-access-rd25z\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.811299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.811226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-dshm\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.811299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.811259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-home\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.811408 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.811348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-model-cache\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.811408 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.811381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40366669-f476-4195-98b5-cd98f85fd8db-tls-certs\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.815868 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.815846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"]
Aug 03 16:50:16.911857 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.911820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-model-cache\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.911866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40366669-f476-4195-98b5-cd98f85fd8db-tls-certs\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.911932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-tmp-dir\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.911968    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd25z\" (UniqueName: \"kubernetes.io/projected/40366669-f476-4195-98b5-cd98f85fd8db-kube-api-access-rd25z\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.911999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-dshm\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912037 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.912030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-home\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912290 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.912246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-model-cache\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912290 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.912279    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-tmp-dir\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.912426 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.912408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-home\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.914119 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.914098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-dshm\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.914426 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.914410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40366669-f476-4195-98b5-cd98f85fd8db-tls-certs\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:16.925255 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:16.925236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd25z\" (UniqueName: \"kubernetes.io/projected/40366669-f476-4195-98b5-cd98f85fd8db-kube-api-access-rd25z\") pod \"auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") " pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:17.098850 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:17.098819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:17.244550 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:17.244510    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"]
Aug 03 16:50:17.248005 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:50:17.247973    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40366669_f476_4195_98b5_cd98f85fd8db.slice/crio-8a8a5c2695e3880a17a4a92f2d523013bd937e4c397af15a03b36a0db8edc298 WatchSource:0}: Error finding container 8a8a5c2695e3880a17a4a92f2d523013bd937e4c397af15a03b36a0db8edc298: Status 404 returned error can't find the container with id 8a8a5c2695e3880a17a4a92f2d523013bd937e4c397af15a03b36a0db8edc298
Aug 03 16:50:17.715901 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:17.715870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt" event={"ID":"40366669-f476-4195-98b5-cd98f85fd8db","Type":"ContainerStarted","Data":"06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f"}
Aug 03 16:50:17.715901 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:17.715909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt" event={"ID":"40366669-f476-4195-98b5-cd98f85fd8db","Type":"ContainerStarted","Data":"8a8a5c2695e3880a17a4a92f2d523013bd937e4c397af15a03b36a0db8edc298"}
Aug 03 16:50:17.737464 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:17.737414    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt" podStartSLOduration=1.737400061 podStartE2EDuration="1.737400061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:50:17.736386265 +0000 UTC m=+2760.003202283" watchObservedRunningTime="2026-08-03 16:50:17.737400061 +0000 UTC m=+2760.004216067"
Aug 03 16:50:19.095948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:19.095910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:50:27.099231 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:27.099203    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:27.099817 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:27.099242    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:27.112486 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:27.112460    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:27.767639 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:27.767612    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:50:46.388717 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:50:46.388681    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:51:11.298346 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:51:11.298311    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:51:18.390195 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:51:18.390163    2564 scope.go:117] "RemoveContainer" containerID="b72745f24908f3f2ad1f8a97fa9c568da562a75a824207f5c99bb1cd55a17c15"
Aug 03 16:51:26.891312 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:51:26.891278    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:51:37.909032 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:51:37.908998    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:51:53.495798 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:51:53.495766    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:52:06.991452 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:52:06.991416    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:52:14.787333 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:52:14.787241    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:52:23.298491 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:52:23.298453    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:52:32.902719 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:52:32.902679    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:52:43.088723 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:52:43.088684    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:52:56.611458 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:52:56.611404    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:53:07.196063 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:53:07.196022    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:53:23.102052 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:53:23.102013    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:53:38.691655 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:53:38.691623    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:53:48.894913 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:53:48.894835    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:54:05.296789 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:05.296756    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:54:18.332390 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:18.332363    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:54:18.341720 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:18.341697    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:54:18.899809 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:18.899777    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:54:23.175398 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:23.175368    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"]
Aug 03 16:54:28.888178 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:28.888147    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:54:38.193296 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.193248    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt" podUID="40366669-f476-4195-98b5-cd98f85fd8db" containerName="main" containerID="cri-o://06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f" gracePeriod=45
Aug 03 16:54:38.435729 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.435700    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:54:38.464612 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.464536    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-dshm\") pod \"40366669-f476-4195-98b5-cd98f85fd8db\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") "
Aug 03 16:54:38.464790 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.464610    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40366669-f476-4195-98b5-cd98f85fd8db-tls-certs\") pod \"40366669-f476-4195-98b5-cd98f85fd8db\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") "
Aug 03 16:54:38.464790 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.464642    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-home\") pod \"40366669-f476-4195-98b5-cd98f85fd8db\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") "
Aug 03 16:54:38.464790 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.464686    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd25z\" (UniqueName: \"kubernetes.io/projected/40366669-f476-4195-98b5-cd98f85fd8db-kube-api-access-rd25z\") pod \"40366669-f476-4195-98b5-cd98f85fd8db\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") "
Aug 03 16:54:38.464790 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.464724    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-tmp-dir\") pod \"40366669-f476-4195-98b5-cd98f85fd8db\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") "
Aug 03 16:54:38.465007 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.464837    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-model-cache\") pod \"40366669-f476-4195-98b5-cd98f85fd8db\" (UID: \"40366669-f476-4195-98b5-cd98f85fd8db\") "
Aug 03 16:54:38.465007 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.464912    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-home" (OuterVolumeSpecName: "home") pod "40366669-f476-4195-98b5-cd98f85fd8db" (UID: "40366669-f476-4195-98b5-cd98f85fd8db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:54:38.465098 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.465040    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40366669-f476-4195-98b5-cd98f85fd8db" (UID: "40366669-f476-4195-98b5-cd98f85fd8db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:54:38.465145 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.465122    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-model-cache" (OuterVolumeSpecName: "model-cache") pod "40366669-f476-4195-98b5-cd98f85fd8db" (UID: "40366669-f476-4195-98b5-cd98f85fd8db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:54:38.465184 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.465162    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:54:38.465224 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.465181    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:54:38.467017 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.466982    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40366669-f476-4195-98b5-cd98f85fd8db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40366669-f476-4195-98b5-cd98f85fd8db" (UID: "40366669-f476-4195-98b5-cd98f85fd8db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:54:38.467268 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.467247    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-dshm" (OuterVolumeSpecName: "dshm") pod "40366669-f476-4195-98b5-cd98f85fd8db" (UID: "40366669-f476-4195-98b5-cd98f85fd8db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:54:38.467329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.467247    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40366669-f476-4195-98b5-cd98f85fd8db-kube-api-access-rd25z" (OuterVolumeSpecName: "kube-api-access-rd25z") pod "40366669-f476-4195-98b5-cd98f85fd8db" (UID: "40366669-f476-4195-98b5-cd98f85fd8db"). InnerVolumeSpecName "kube-api-access-rd25z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:54:38.468646 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.468624    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"]
Aug 03 16:54:38.468978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.468964    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40366669-f476-4195-98b5-cd98f85fd8db" containerName="main"
Aug 03 16:54:38.468978 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.468979    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="40366669-f476-4195-98b5-cd98f85fd8db" containerName="main"
Aug 03 16:54:38.469213 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.469032    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="40366669-f476-4195-98b5-cd98f85fd8db" containerName="main"
Aug 03 16:54:38.472115 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.471930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.474697 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.474677    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"sub-test-model-0fae8e-kserve-self-signed-certs\""
Aug 03 16:54:38.482466 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.482445    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"]
Aug 03 16:54:38.566027 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.565998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-tmp-dir\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.566027 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/926a2585-76a1-49c0-82a0-b338b9a19c92-tls-certs\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.566246 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x2c7\" (UniqueName: \"kubernetes.io/projected/926a2585-76a1-49c0-82a0-b338b9a19c92-kube-api-access-6x2c7\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.566246 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566074    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-home\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.566246 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-dshm\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.566246 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-model-cache\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.566451 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566251    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:54:38.566451 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566276    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40366669-f476-4195-98b5-cd98f85fd8db-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:54:38.566451 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566290    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40366669-f476-4195-98b5-cd98f85fd8db-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:54:38.566451 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.566303    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd25z\" (UniqueName: \"kubernetes.io/projected/40366669-f476-4195-98b5-cd98f85fd8db-kube-api-access-rd25z\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:54:38.666669 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.666634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-model-cache\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.666895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.666709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-tmp-dir\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.666895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.666728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/926a2585-76a1-49c0-82a0-b338b9a19c92-tls-certs\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.666895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.666772    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x2c7\" (UniqueName: \"kubernetes.io/projected/926a2585-76a1-49c0-82a0-b338b9a19c92-kube-api-access-6x2c7\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.666895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.666801    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-home\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.666895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.666823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-dshm\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.667191 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.667114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-model-cache\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.667191 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.667125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-tmp-dir\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.667299 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.667243    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-home\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.669093 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.669071    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-dshm\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.669675 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.669652    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/926a2585-76a1-49c0-82a0-b338b9a19c92-tls-certs\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.675884 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.675860    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x2c7\" (UniqueName: \"kubernetes.io/projected/926a2585-76a1-49c0-82a0-b338b9a19c92-kube-api-access-6x2c7\") pod \"sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") " pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.788926 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.788847    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:38.849552 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.849446    2564 generic.go:356] "Generic (PLEG): container finished" podID="40366669-f476-4195-98b5-cd98f85fd8db" containerID="06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f" exitCode=0
Aug 03 16:54:38.849700 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.849570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt" event={"ID":"40366669-f476-4195-98b5-cd98f85fd8db","Type":"ContainerDied","Data":"06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f"}
Aug 03 16:54:38.849700 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.849608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt" event={"ID":"40366669-f476-4195-98b5-cd98f85fd8db","Type":"ContainerDied","Data":"8a8a5c2695e3880a17a4a92f2d523013bd937e4c397af15a03b36a0db8edc298"}
Aug 03 16:54:38.849700 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.849628    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"
Aug 03 16:54:38.849895 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.849628    2564 scope.go:117] "RemoveContainer" containerID="06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f"
Aug 03 16:54:38.860434 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.860411    2564 scope.go:117] "RemoveContainer" containerID="06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f"
Aug 03 16:54:38.860895 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:54:38.860866    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f\": container with ID starting with 06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f not found: ID does not exist" containerID="06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f"
Aug 03 16:54:38.860965 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.860909    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f"} err="failed to get container status \"06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f\": rpc error: code = NotFound desc = could not find container \"06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f\": container with ID starting with 06f44fefd126c67b5cd46bf4261b67bc226b46bd7eafd69eab402b04e9622b1f not found: ID does not exist"
Aug 03 16:54:38.876553 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.876508    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"]
Aug 03 16:54:38.880184 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.880162    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/auth-test-model-0fae8e-kserve-6f9dfd8955-bppjt"]
Aug 03 16:54:38.923110 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:38.923083    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"]
Aug 03 16:54:38.926191 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:54:38.926169    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod926a2585_76a1_49c0_82a0_b338b9a19c92.slice/crio-a0c6935ef23b6a9bde2f5bbabe27504d939ae11b0b116e58f31cfcc31130ab8e WatchSource:0}: Error finding container a0c6935ef23b6a9bde2f5bbabe27504d939ae11b0b116e58f31cfcc31130ab8e: Status 404 returned error can't find the container with id a0c6935ef23b6a9bde2f5bbabe27504d939ae11b0b116e58f31cfcc31130ab8e
Aug 03 16:54:39.857139 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:39.857094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp" event={"ID":"926a2585-76a1-49c0-82a0-b338b9a19c92","Type":"ContainerStarted","Data":"f0163a7b5524e862169466648686e853b52caf0e6ebc4b478c01edbc407364a7"}
Aug 03 16:54:39.857139 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:39.857141    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp" event={"ID":"926a2585-76a1-49c0-82a0-b338b9a19c92","Type":"ContainerStarted","Data":"a0c6935ef23b6a9bde2f5bbabe27504d939ae11b0b116e58f31cfcc31130ab8e"}
Aug 03 16:54:39.875720 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:39.875675    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp" podStartSLOduration=1.875661268 podStartE2EDuration="1.875661268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:54:39.874458803 +0000 UTC m=+3022.141274810" watchObservedRunningTime="2026-08-03 16:54:39.875661268 +0000 UTC m=+3022.142477286"
Aug 03 16:54:40.243888 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:40.243812    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40366669-f476-4195-98b5-cd98f85fd8db" path="/var/lib/kubelet/pods/40366669-f476-4195-98b5-cd98f85fd8db/volumes"
Aug 03 16:54:40.993218 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:40.993183    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:54:48.789562 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:48.789502    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:48.789562 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:48.789575    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:48.802181 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:48.802147    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:54:48.909869 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:54:48.909836    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:55:08.645396 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.645359    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"]
Aug 03 16:55:08.648512 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.648494    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.651650 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.651631    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"kube-root-ca.crt\""
Aug 03 16:55:08.652942 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.652925    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"openshift-service-ca.crt\""
Aug 03 16:55:08.653024 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.652963    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"sub-test-model-7055f0-kserve-self-signed-certs\""
Aug 03 16:55:08.653084 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.653067    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"default-dockercfg-cwtm8\""
Aug 03 16:55:08.658083 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.658060    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"]
Aug 03 16:55:08.733780 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.733726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-dshm\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.733919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.733790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-tmp-dir\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.733919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.733867    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-home\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.733919 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.733906    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-model-cache\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.734041 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.733943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twnb7\" (UniqueName: \"kubernetes.io/projected/928b165d-029b-4670-bbc4-6bf356ac8585-kube-api-access-twnb7\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.734041 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.733986    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/928b165d-029b-4670-bbc4-6bf356ac8585-tls-certs\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.834864 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.834833    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/928b165d-029b-4670-bbc4-6bf356ac8585-tls-certs\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835026 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.834872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-dshm\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835026 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.834910    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-tmp-dir\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835121 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.835017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-home\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835121 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.835069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-model-cache\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835121 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.835110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twnb7\" (UniqueName: \"kubernetes.io/projected/928b165d-029b-4670-bbc4-6bf356ac8585-kube-api-access-twnb7\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835343 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.835318    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-tmp-dir\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835477 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.835375    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-home\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.835477 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.835457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-model-cache\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.837055 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.837037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-dshm\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.837323 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.837307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/928b165d-029b-4670-bbc4-6bf356ac8585-tls-certs\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.846136 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.846113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twnb7\" (UniqueName: \"kubernetes.io/projected/928b165d-029b-4670-bbc4-6bf356ac8585-kube-api-access-twnb7\") pod \"sub-test-model-7055f0-kserve-6f458df757-mkmfh\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") " pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:08.958566 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:08.958484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:09.083706 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:09.083675    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"]
Aug 03 16:55:09.086466 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:55:09.086442    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod928b165d_029b_4670_bbc4_6bf356ac8585.slice/crio-12299053074b7abd0a5e5763d9ef20a57f96d5200e299a3a3fa6cec7f717c07a WatchSource:0}: Error finding container 12299053074b7abd0a5e5763d9ef20a57f96d5200e299a3a3fa6cec7f717c07a: Status 404 returned error can't find the container with id 12299053074b7abd0a5e5763d9ef20a57f96d5200e299a3a3fa6cec7f717c07a
Aug 03 16:55:09.990503 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:09.990461    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh" event={"ID":"928b165d-029b-4670-bbc4-6bf356ac8585","Type":"ContainerStarted","Data":"c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6"}
Aug 03 16:55:09.990503 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:09.990509    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh" event={"ID":"928b165d-029b-4670-bbc4-6bf356ac8585","Type":"ContainerStarted","Data":"12299053074b7abd0a5e5763d9ef20a57f96d5200e299a3a3fa6cec7f717c07a"}
Aug 03 16:55:10.014354 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:10.014255    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh" podStartSLOduration=2.014235402 podStartE2EDuration="2.014235402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:55:10.012415513 +0000 UTC m=+3052.279231522" watchObservedRunningTime="2026-08-03 16:55:10.014235402 +0000 UTC m=+3052.281051409"
Aug 03 16:55:11.305710 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:11.305678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:55:18.426294 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:18.426263    2564 scope.go:117] "RemoveContainer" containerID="089cbd2066c6634b7f9d17c48d7c3955942cf0648291f68d730e7cc0bba899b5"
Aug 03 16:55:18.958778 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:18.958730    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:18.959001 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:18.958795    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:18.971583 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:18.971556    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:19.040131 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:19.040100    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:55:51.490148 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:55:51.490109    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:56:04.088508 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:04.088471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:56:16.694565 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:16.694531    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:56:22.155613 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:22.155579    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"]
Aug 03 16:56:23.048653 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:23.048623    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"]
Aug 03 16:56:25.691383 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:25.691345    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:56:35.598567 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:35.598531    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:56:37.174415 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.174370    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp" podUID="926a2585-76a1-49c0-82a0-b338b9a19c92" containerName="main" containerID="cri-o://f0163a7b5524e862169466648686e853b52caf0e6ebc4b478c01edbc407364a7" gracePeriod=45
Aug 03 16:56:37.363097 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.363058    2564 generic.go:356] "Generic (PLEG): container finished" podID="926a2585-76a1-49c0-82a0-b338b9a19c92" containerID="f0163a7b5524e862169466648686e853b52caf0e6ebc4b478c01edbc407364a7" exitCode=0
Aug 03 16:56:37.363255 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.363123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp" event={"ID":"926a2585-76a1-49c0-82a0-b338b9a19c92","Type":"ContainerDied","Data":"f0163a7b5524e862169466648686e853b52caf0e6ebc4b478c01edbc407364a7"}
Aug 03 16:56:37.413672 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.413644    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:56:37.513546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513459    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-model-cache\") pod \"926a2585-76a1-49c0-82a0-b338b9a19c92\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") "
Aug 03 16:56:37.513546 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513496    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-tmp-dir\") pod \"926a2585-76a1-49c0-82a0-b338b9a19c92\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") "
Aug 03 16:56:37.513771 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513552    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-dshm\") pod \"926a2585-76a1-49c0-82a0-b338b9a19c92\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") "
Aug 03 16:56:37.513771 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513627    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x2c7\" (UniqueName: \"kubernetes.io/projected/926a2585-76a1-49c0-82a0-b338b9a19c92-kube-api-access-6x2c7\") pod \"926a2585-76a1-49c0-82a0-b338b9a19c92\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") "
Aug 03 16:56:37.513771 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513659    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/926a2585-76a1-49c0-82a0-b338b9a19c92-tls-certs\") pod \"926a2585-76a1-49c0-82a0-b338b9a19c92\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") "
Aug 03 16:56:37.513771 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513681    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-home\") pod \"926a2585-76a1-49c0-82a0-b338b9a19c92\" (UID: \"926a2585-76a1-49c0-82a0-b338b9a19c92\") "
Aug 03 16:56:37.513981 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513781    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-model-cache" (OuterVolumeSpecName: "model-cache") pod "926a2585-76a1-49c0-82a0-b338b9a19c92" (UID: "926a2585-76a1-49c0-82a0-b338b9a19c92"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:37.513981 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.513853    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "926a2585-76a1-49c0-82a0-b338b9a19c92" (UID: "926a2585-76a1-49c0-82a0-b338b9a19c92"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:37.514100 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.514012    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:37.514100 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.514032    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:37.514100 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.514043    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-home" (OuterVolumeSpecName: "home") pod "926a2585-76a1-49c0-82a0-b338b9a19c92" (UID: "926a2585-76a1-49c0-82a0-b338b9a19c92"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:37.515870 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.515847    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-dshm" (OuterVolumeSpecName: "dshm") pod "926a2585-76a1-49c0-82a0-b338b9a19c92" (UID: "926a2585-76a1-49c0-82a0-b338b9a19c92"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:37.515870 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.515853    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926a2585-76a1-49c0-82a0-b338b9a19c92-kube-api-access-6x2c7" (OuterVolumeSpecName: "kube-api-access-6x2c7") pod "926a2585-76a1-49c0-82a0-b338b9a19c92" (UID: "926a2585-76a1-49c0-82a0-b338b9a19c92"). InnerVolumeSpecName "kube-api-access-6x2c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:56:37.516008 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.515887    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/926a2585-76a1-49c0-82a0-b338b9a19c92-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "926a2585-76a1-49c0-82a0-b338b9a19c92" (UID: "926a2585-76a1-49c0-82a0-b338b9a19c92"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:56:37.614875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.614841    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/926a2585-76a1-49c0-82a0-b338b9a19c92-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:37.614875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.614870    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:37.614875 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.614880    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/926a2585-76a1-49c0-82a0-b338b9a19c92-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:37.615123 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:37.614892    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x2c7\" (UniqueName: \"kubernetes.io/projected/926a2585-76a1-49c0-82a0-b338b9a19c92-kube-api-access-6x2c7\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:38.069324 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.069281    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh" podUID="928b165d-029b-4670-bbc4-6bf356ac8585" containerName="main" containerID="cri-o://c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6" gracePeriod=45
Aug 03 16:56:38.308160 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.308134    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:56:38.369288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.369205    2564 generic.go:356] "Generic (PLEG): container finished" podID="928b165d-029b-4670-bbc4-6bf356ac8585" containerID="c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6" exitCode=0
Aug 03 16:56:38.369288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.369264    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"
Aug 03 16:56:38.369534 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.369292    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh" event={"ID":"928b165d-029b-4670-bbc4-6bf356ac8585","Type":"ContainerDied","Data":"c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6"}
Aug 03 16:56:38.369534 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.369340    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh" event={"ID":"928b165d-029b-4670-bbc4-6bf356ac8585","Type":"ContainerDied","Data":"12299053074b7abd0a5e5763d9ef20a57f96d5200e299a3a3fa6cec7f717c07a"}
Aug 03 16:56:38.369534 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.369363    2564 scope.go:117] "RemoveContainer" containerID="c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6"
Aug 03 16:56:38.370663 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.370647    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"
Aug 03 16:56:38.370663 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.370652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp" event={"ID":"926a2585-76a1-49c0-82a0-b338b9a19c92","Type":"ContainerDied","Data":"a0c6935ef23b6a9bde2f5bbabe27504d939ae11b0b116e58f31cfcc31130ab8e"}
Aug 03 16:56:38.379627 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.379611    2564 scope.go:117] "RemoveContainer" containerID="c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6"
Aug 03 16:56:38.379913 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:56:38.379893    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6\": container with ID starting with c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6 not found: ID does not exist" containerID="c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6"
Aug 03 16:56:38.379981 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.379919    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6"} err="failed to get container status \"c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6\": rpc error: code = NotFound desc = could not find container \"c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6\": container with ID starting with c3f609879dbe3928f790f7ffe0c76388d013707d387f53237c14f0ac2df4aae6 not found: ID does not exist"
Aug 03 16:56:38.379981 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.379937    2564 scope.go:117] "RemoveContainer" containerID="f0163a7b5524e862169466648686e853b52caf0e6ebc4b478c01edbc407364a7"
Aug 03 16:56:38.387384 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.387367    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"]
Aug 03 16:56:38.391755 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.391711    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/sub-test-model-0fae8e-kserve-5665fd5c9c-r89lp"]
Aug 03 16:56:38.422507 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422480    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-tmp-dir\") pod \"928b165d-029b-4670-bbc4-6bf356ac8585\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") "
Aug 03 16:56:38.422658 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422517    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-model-cache\") pod \"928b165d-029b-4670-bbc4-6bf356ac8585\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") "
Aug 03 16:56:38.422658 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422552    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twnb7\" (UniqueName: \"kubernetes.io/projected/928b165d-029b-4670-bbc4-6bf356ac8585-kube-api-access-twnb7\") pod \"928b165d-029b-4670-bbc4-6bf356ac8585\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") "
Aug 03 16:56:38.422658 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422571    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-home\") pod \"928b165d-029b-4670-bbc4-6bf356ac8585\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") "
Aug 03 16:56:38.422658 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422608    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-dshm\") pod \"928b165d-029b-4670-bbc4-6bf356ac8585\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") "
Aug 03 16:56:38.422658 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422633    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/928b165d-029b-4670-bbc4-6bf356ac8585-tls-certs\") pod \"928b165d-029b-4670-bbc4-6bf356ac8585\" (UID: \"928b165d-029b-4670-bbc4-6bf356ac8585\") "
Aug 03 16:56:38.422966 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-model-cache" (OuterVolumeSpecName: "model-cache") pod "928b165d-029b-4670-bbc4-6bf356ac8585" (UID: "928b165d-029b-4670-bbc4-6bf356ac8585"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:38.422966 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422849    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "928b165d-029b-4670-bbc4-6bf356ac8585" (UID: "928b165d-029b-4670-bbc4-6bf356ac8585"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:38.422966 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422883    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-home" (OuterVolumeSpecName: "home") pod "928b165d-029b-4670-bbc4-6bf356ac8585" (UID: "928b165d-029b-4670-bbc4-6bf356ac8585"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:38.423109 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.422989    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:38.423109 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.423017    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:38.423109 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.423031    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:38.424889 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.424867    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-dshm" (OuterVolumeSpecName: "dshm") pod "928b165d-029b-4670-bbc4-6bf356ac8585" (UID: "928b165d-029b-4670-bbc4-6bf356ac8585"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:56:38.424955 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.424930    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/928b165d-029b-4670-bbc4-6bf356ac8585-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "928b165d-029b-4670-bbc4-6bf356ac8585" (UID: "928b165d-029b-4670-bbc4-6bf356ac8585"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:56:38.424991 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.424939    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928b165d-029b-4670-bbc4-6bf356ac8585-kube-api-access-twnb7" (OuterVolumeSpecName: "kube-api-access-twnb7") pod "928b165d-029b-4670-bbc4-6bf356ac8585" (UID: "928b165d-029b-4670-bbc4-6bf356ac8585"). InnerVolumeSpecName "kube-api-access-twnb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:56:38.523476 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.523439    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/928b165d-029b-4670-bbc4-6bf356ac8585-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:38.523476 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.523469    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/928b165d-029b-4670-bbc4-6bf356ac8585-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:38.523476 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.523481    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twnb7\" (UniqueName: \"kubernetes.io/projected/928b165d-029b-4670-bbc4-6bf356ac8585-kube-api-access-twnb7\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:56:38.688747 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:56:38.688684    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod928b165d_029b_4670_bbc4_6bf356ac8585.slice/crio-12299053074b7abd0a5e5763d9ef20a57f96d5200e299a3a3fa6cec7f717c07a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod928b165d_029b_4670_bbc4_6bf356ac8585.slice\": RecentStats: unable to find data in memory cache]"
Aug 03 16:56:38.688920 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:56:38.688795    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod928b165d_029b_4670_bbc4_6bf356ac8585.slice\": RecentStats: unable to find data in memory cache]"
Aug 03 16:56:38.692619 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.692586    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"]
Aug 03 16:56:38.696719 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:38.696694    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/sub-test-model-7055f0-kserve-6f458df757-mkmfh"]
Aug 03 16:56:40.243066 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:40.242983    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="926a2585-76a1-49c0-82a0-b338b9a19c92" path="/var/lib/kubelet/pods/926a2585-76a1-49c0-82a0-b338b9a19c92/volumes"
Aug 03 16:56:40.243469 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:40.243378    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="928b165d-029b-4670-bbc4-6bf356ac8585" path="/var/lib/kubelet/pods/928b165d-029b-4670-bbc4-6bf356ac8585/volumes"
Aug 03 16:56:52.363024 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.362991    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"]
Aug 03 16:56:52.363429 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.363289    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="926a2585-76a1-49c0-82a0-b338b9a19c92" containerName="main"
Aug 03 16:56:52.363429 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.363299    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="926a2585-76a1-49c0-82a0-b338b9a19c92" containerName="main"
Aug 03 16:56:52.363429 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.363313    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928b165d-029b-4670-bbc4-6bf356ac8585" containerName="main"
Aug 03 16:56:52.363429 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.363318    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="928b165d-029b-4670-bbc4-6bf356ac8585" containerName="main"
Aug 03 16:56:52.363429 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.363369    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="928b165d-029b-4670-bbc4-6bf356ac8585" containerName="main"
Aug 03 16:56:52.363429 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.363378    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="926a2585-76a1-49c0-82a0-b338b9a19c92" containerName="main"
Aug 03 16:56:52.367922 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.367906    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.371979 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.371956    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"default-dockercfg-cwtm8\""
Aug 03 16:56:52.372131 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.371989    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"kube-root-ca.crt\""
Aug 03 16:56:52.372131 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.371963    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"openshift-service-ca.crt\""
Aug 03 16:56:52.372131 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.372043    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"rate-test-model-7055f0-kserve-self-signed-certs\""
Aug 03 16:56:52.378316 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.378296    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"]
Aug 03 16:56:52.549616 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.549575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-home\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.549815 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.549635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-model-cache\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.549815 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.549659    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-dshm\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.549815 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.549721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tmp-dir\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.549815 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.549790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tls-certs\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.549962 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.549819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6fpw\" (UniqueName: \"kubernetes.io/projected/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-kube-api-access-n6fpw\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651127 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tls-certs\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651127 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651083    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6fpw\" (UniqueName: \"kubernetes.io/projected/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-kube-api-access-n6fpw\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651127 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-home\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651127 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-model-cache\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651451 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651144    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-dshm\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651451 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tmp-dir\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651565 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-home\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651622 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-model-cache\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.651681 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.651617    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tmp-dir\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.653480 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.653458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-dshm\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.653774 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.653726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tls-certs\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.660556 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.660530    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6fpw\" (UniqueName: \"kubernetes.io/projected/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-kube-api-access-n6fpw\") pod \"rate-test-model-7055f0-kserve-5576dc966c-4ptjw\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") " pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.677297 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.677270    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:56:52.808669 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:52.808642    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"]
Aug 03 16:56:52.810421 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:56:52.810396    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb35f969_ad34_49f9_abe5_e40a1d1b1fe5.slice/crio-2fb3ab274e00730b032efc248a2d3bf143ed3cbf6d87494d71743b4df59f3222 WatchSource:0}: Error finding container 2fb3ab274e00730b032efc248a2d3bf143ed3cbf6d87494d71743b4df59f3222: Status 404 returned error can't find the container with id 2fb3ab274e00730b032efc248a2d3bf143ed3cbf6d87494d71743b4df59f3222
Aug 03 16:56:53.435941 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:53.435904    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw" event={"ID":"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5","Type":"ContainerStarted","Data":"80d32711347b5e632525a8e737d915c21d4945efd6939a1ca308af2b42071909"}
Aug 03 16:56:53.436331 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:53.435948    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw" event={"ID":"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5","Type":"ContainerStarted","Data":"2fb3ab274e00730b032efc248a2d3bf143ed3cbf6d87494d71743b4df59f3222"}
Aug 03 16:56:53.453631 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:53.453582    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw" podStartSLOduration=1.453565567 podStartE2EDuration="1.453565567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:56:53.452859383 +0000 UTC m=+3155.719675391" watchObservedRunningTime="2026-08-03 16:56:53.453565567 +0000 UTC m=+3155.720381574"
Aug 03 16:56:54.702492 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:56:54.702460    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:57:02.677903 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:57:02.677867    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:57:02.678302 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:57:02.677919    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:57:02.690516 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:57:02.690490    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:57:03.489291 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:57:03.489258    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:57:40.386930 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:57:40.386895    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:57:48.093380 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:57:48.093346    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:58:20.389866 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:20.389784    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:58:26.746763 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:26.746715    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"]
Aug 03 16:58:27.758975 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:27.758938    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7305-140502673858992"]
Aug 03 16:58:27.762172 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:27.762154    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:27.765275 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:27.765259    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-d468t\""
Aug 03 16:58:27.769885 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:27.769864    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673858992"]
Aug 03 16:58:27.889751 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:27.889706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd929\" (UniqueName: \"kubernetes.io/projected/afafcc3a-2923-4624-821d-2086ebb6bd5e-kube-api-access-pd929\") pod \"test-curl-7305-140502673858992\" (UID: \"afafcc3a-2923-4624-821d-2086ebb6bd5e\") " pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:27.990635 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:27.990601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd929\" (UniqueName: \"kubernetes.io/projected/afafcc3a-2923-4624-821d-2086ebb6bd5e-kube-api-access-pd929\") pod \"test-curl-7305-140502673858992\" (UID: \"afafcc3a-2923-4624-821d-2086ebb6bd5e\") " pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:28.000351 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:28.000318    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd929\" (UniqueName: \"kubernetes.io/projected/afafcc3a-2923-4624-821d-2086ebb6bd5e-kube-api-access-pd929\") pod \"test-curl-7305-140502673858992\" (UID: \"afafcc3a-2923-4624-821d-2086ebb6bd5e\") " pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:28.072490 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:28.072450    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:28.203341 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:28.203315    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673858992"]
Aug 03 16:58:28.205601 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:58:28.205569    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafafcc3a_2923_4624_821d_2086ebb6bd5e.slice/crio-e9eb29a81f539757781052cae55602c785771f4724ce9e44fdcc800a182a98da WatchSource:0}: Error finding container e9eb29a81f539757781052cae55602c785771f4724ce9e44fdcc800a182a98da: Status 404 returned error can't find the container with id e9eb29a81f539757781052cae55602c785771f4724ce9e44fdcc800a182a98da
Aug 03 16:58:28.206889 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:28.206872    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:58:28.839445 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:28.839411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673858992" event={"ID":"afafcc3a-2923-4624-821d-2086ebb6bd5e","Type":"ContainerStarted","Data":"e9eb29a81f539757781052cae55602c785771f4724ce9e44fdcc800a182a98da"}
Aug 03 16:58:29.087587 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:29.087556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:58:29.844816 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:29.844785    2564 generic.go:356] "Generic (PLEG): container finished" podID="afafcc3a-2923-4624-821d-2086ebb6bd5e" containerID="4cb8a9b15434088fde119bc63ac7063582ae7ba3b6ffea07863f87f049d53dba" exitCode=0
Aug 03 16:58:29.845215 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:29.844873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673858992" event={"ID":"afafcc3a-2923-4624-821d-2086ebb6bd5e","Type":"ContainerDied","Data":"4cb8a9b15434088fde119bc63ac7063582ae7ba3b6ffea07863f87f049d53dba"}
Aug 03 16:58:30.976165 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:30.976134    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:31.013009 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.012981    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7305-140502673858992_afafcc3a-2923-4624-821d-2086ebb6bd5e/test-curl-7305-140502673858992/0.log"
Aug 03 16:58:31.038225 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.038188    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7305-140502673858992"]
Aug 03 16:58:31.040245 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.040219    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7305-140502673858992"]
Aug 03 16:58:31.124345 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.124256    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd929\" (UniqueName: \"kubernetes.io/projected/afafcc3a-2923-4624-821d-2086ebb6bd5e-kube-api-access-pd929\") pod \"afafcc3a-2923-4624-821d-2086ebb6bd5e\" (UID: \"afafcc3a-2923-4624-821d-2086ebb6bd5e\") "
Aug 03 16:58:31.126439 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.126407    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afafcc3a-2923-4624-821d-2086ebb6bd5e-kube-api-access-pd929" (OuterVolumeSpecName: "kube-api-access-pd929") pod "afafcc3a-2923-4624-821d-2086ebb6bd5e" (UID: "afafcc3a-2923-4624-821d-2086ebb6bd5e"). InnerVolumeSpecName "kube-api-access-pd929". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:58:31.161712 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.161674    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7305-140502673861808"]
Aug 03 16:58:31.162039 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.162024    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afafcc3a-2923-4624-821d-2086ebb6bd5e" containerName="test-curl-7305-140502673858992"
Aug 03 16:58:31.162091 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.162041    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="afafcc3a-2923-4624-821d-2086ebb6bd5e" containerName="test-curl-7305-140502673858992"
Aug 03 16:58:31.162130 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.162112    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="afafcc3a-2923-4624-821d-2086ebb6bd5e" containerName="test-curl-7305-140502673858992"
Aug 03 16:58:31.165062 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.165046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673861808"
Aug 03 16:58:31.170154 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.170127    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673861808"]
Aug 03 16:58:31.225089 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.225053    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd929\" (UniqueName: \"kubernetes.io/projected/afafcc3a-2923-4624-821d-2086ebb6bd5e-kube-api-access-pd929\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:31.326004 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.325963    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg59v\" (UniqueName: \"kubernetes.io/projected/2e58c773-848a-4744-ad43-7c3e9de95924-kube-api-access-lg59v\") pod \"test-curl-7305-140502673861808\" (UID: \"2e58c773-848a-4744-ad43-7c3e9de95924\") " pod="opendatahub/test-curl-7305-140502673861808"
Aug 03 16:58:31.427217 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.427115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg59v\" (UniqueName: \"kubernetes.io/projected/2e58c773-848a-4744-ad43-7c3e9de95924-kube-api-access-lg59v\") pod \"test-curl-7305-140502673861808\" (UID: \"2e58c773-848a-4744-ad43-7c3e9de95924\") " pod="opendatahub/test-curl-7305-140502673861808"
Aug 03 16:58:31.437044 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.437019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg59v\" (UniqueName: \"kubernetes.io/projected/2e58c773-848a-4744-ad43-7c3e9de95924-kube-api-access-lg59v\") pod \"test-curl-7305-140502673861808\" (UID: \"2e58c773-848a-4744-ad43-7c3e9de95924\") " pod="opendatahub/test-curl-7305-140502673861808"
Aug 03 16:58:31.475093 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.475058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673861808"
Aug 03 16:58:31.603440 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.603408    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673861808"]
Aug 03 16:58:31.606804 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:58:31.606777    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e58c773_848a_4744_ad43_7c3e9de95924.slice/crio-ccd82b0f215dba42eb891038aa1ff4f5c968883965764d17d82fec6def7fb567 WatchSource:0}: Error finding container ccd82b0f215dba42eb891038aa1ff4f5c968883965764d17d82fec6def7fb567: Status 404 returned error can't find the container with id ccd82b0f215dba42eb891038aa1ff4f5c968883965764d17d82fec6def7fb567
Aug 03 16:58:31.854796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.854756    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:31.854796 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.854778    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9eb29a81f539757781052cae55602c785771f4724ce9e44fdcc800a182a98da"
Aug 03 16:58:31.855866 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:31.855843    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673861808" event={"ID":"2e58c773-848a-4744-ad43-7c3e9de95924","Type":"ContainerStarted","Data":"ccd82b0f215dba42eb891038aa1ff4f5c968883965764d17d82fec6def7fb567"}
Aug 03 16:58:31.865272 ip-10-0-138-110 kubenswrapper[2564]: E0803 16:58:31.865247    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7305-140502673858992\" is forbidden: User \"system:node:ip-10-0-138-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-110.ec2.internal' and this object" podUID="afafcc3a-2923-4624-821d-2086ebb6bd5e" pod="opendatahub/test-curl-7305-140502673858992"
Aug 03 16:58:32.243896 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:32.243814    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afafcc3a-2923-4624-821d-2086ebb6bd5e" path="/var/lib/kubelet/pods/afafcc3a-2923-4624-821d-2086ebb6bd5e/volumes"
Aug 03 16:58:32.861894 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:32.861862    2564 generic.go:356] "Generic (PLEG): container finished" podID="2e58c773-848a-4744-ad43-7c3e9de95924" containerID="dc2b8d5864e414378997c96c4c12575808f50e24a6a65f98c817c78d4bbaea54" exitCode=0
Aug 03 16:58:32.862077 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:32.861919    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673861808" event={"ID":"2e58c773-848a-4744-ad43-7c3e9de95924","Type":"ContainerDied","Data":"dc2b8d5864e414378997c96c4c12575808f50e24a6a65f98c817c78d4bbaea54"}
Aug 03 16:58:33.993498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:33.993472    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673861808"
Aug 03 16:58:34.016577 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.016547    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7305-140502673861808_2e58c773-848a-4744-ad43-7c3e9de95924/test-curl-7305-140502673861808/0.log"
Aug 03 16:58:34.040029 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.039988    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7305-140502673861808"]
Aug 03 16:58:34.044231 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.044207    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7305-140502673861808"]
Aug 03 16:58:34.151885 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.151776    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg59v\" (UniqueName: \"kubernetes.io/projected/2e58c773-848a-4744-ad43-7c3e9de95924-kube-api-access-lg59v\") pod \"2e58c773-848a-4744-ad43-7c3e9de95924\" (UID: \"2e58c773-848a-4744-ad43-7c3e9de95924\") "
Aug 03 16:58:34.153989 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.153956    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e58c773-848a-4744-ad43-7c3e9de95924-kube-api-access-lg59v" (OuterVolumeSpecName: "kube-api-access-lg59v") pod "2e58c773-848a-4744-ad43-7c3e9de95924" (UID: "2e58c773-848a-4744-ad43-7c3e9de95924"). InnerVolumeSpecName "kube-api-access-lg59v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:58:34.227632 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.227596    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:34.227964 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.227950    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e58c773-848a-4744-ad43-7c3e9de95924" containerName="test-curl-7305-140502673861808"
Aug 03 16:58:34.228016 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.227965    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e58c773-848a-4744-ad43-7c3e9de95924" containerName="test-curl-7305-140502673861808"
Aug 03 16:58:34.228059 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.228024    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e58c773-848a-4744-ad43-7c3e9de95924" containerName="test-curl-7305-140502673861808"
Aug 03 16:58:34.231377 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.231362    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:34.243696 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.243671    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e58c773-848a-4744-ad43-7c3e9de95924" path="/var/lib/kubelet/pods/2e58c773-848a-4744-ad43-7c3e9de95924/volumes"
Aug 03 16:58:34.244034 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.244021    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:34.252943 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.252919    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg59v\" (UniqueName: \"kubernetes.io/projected/2e58c773-848a-4744-ad43-7c3e9de95924-kube-api-access-lg59v\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:34.353666 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.353630    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4frcw\" (UniqueName: \"kubernetes.io/projected/d1d3f8b4-b1ba-4d61-b477-62c21deb4556-kube-api-access-4frcw\") pod \"test-curl-7305-140502673976608\" (UID: \"d1d3f8b4-b1ba-4d61-b477-62c21deb4556\") " pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:34.454842 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.454732    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4frcw\" (UniqueName: \"kubernetes.io/projected/d1d3f8b4-b1ba-4d61-b477-62c21deb4556-kube-api-access-4frcw\") pod \"test-curl-7305-140502673976608\" (UID: \"d1d3f8b4-b1ba-4d61-b477-62c21deb4556\") " pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:34.463883 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.463853    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4frcw\" (UniqueName: \"kubernetes.io/projected/d1d3f8b4-b1ba-4d61-b477-62c21deb4556-kube-api-access-4frcw\") pod \"test-curl-7305-140502673976608\" (UID: \"d1d3f8b4-b1ba-4d61-b477-62c21deb4556\") " pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:34.540689 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.540653    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:34.667470 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.667444    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:34.669216 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:58:34.669191    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1d3f8b4_b1ba_4d61_b477_62c21deb4556.slice/crio-5aa44693d0fb47f993b5f7310489df7ac904c493270284726510359fca61bc45 WatchSource:0}: Error finding container 5aa44693d0fb47f993b5f7310489df7ac904c493270284726510359fca61bc45: Status 404 returned error can't find the container with id 5aa44693d0fb47f993b5f7310489df7ac904c493270284726510359fca61bc45
Aug 03 16:58:34.871307 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.871272    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673976608" event={"ID":"d1d3f8b4-b1ba-4d61-b477-62c21deb4556","Type":"ContainerStarted","Data":"5aa44693d0fb47f993b5f7310489df7ac904c493270284726510359fca61bc45"}
Aug 03 16:58:34.872353 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.872334    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673861808"
Aug 03 16:58:34.872452 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:34.872333    2564 scope.go:117] "RemoveContainer" containerID="dc2b8d5864e414378997c96c4c12575808f50e24a6a65f98c817c78d4bbaea54"
Aug 03 16:58:35.879166 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:35.879136    2564 generic.go:356] "Generic (PLEG): container finished" podID="d1d3f8b4-b1ba-4d61-b477-62c21deb4556" containerID="e8d2ce8cf9dac52815545ea0af731c77f297ec03b52b941a3c7fa2307a05389d" exitCode=0
Aug 03 16:58:35.879505 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:35.879177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673976608" event={"ID":"d1d3f8b4-b1ba-4d61-b477-62c21deb4556","Type":"ContainerDied","Data":"e8d2ce8cf9dac52815545ea0af731c77f297ec03b52b941a3c7fa2307a05389d"}
Aug 03 16:58:37.009245 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.009221    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:37.031226 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.031203    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7305-140502673976608_d1d3f8b4-b1ba-4d61-b477-62c21deb4556/test-curl-7305-140502673976608/0.log"
Aug 03 16:58:37.056981 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.056943    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:37.060776 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.060752    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:37.180080 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.180012    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4frcw\" (UniqueName: \"kubernetes.io/projected/d1d3f8b4-b1ba-4d61-b477-62c21deb4556-kube-api-access-4frcw\") pod \"d1d3f8b4-b1ba-4d61-b477-62c21deb4556\" (UID: \"d1d3f8b4-b1ba-4d61-b477-62c21deb4556\") "
Aug 03 16:58:37.182491 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.182468    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d3f8b4-b1ba-4d61-b477-62c21deb4556-kube-api-access-4frcw" (OuterVolumeSpecName: "kube-api-access-4frcw") pod "d1d3f8b4-b1ba-4d61-b477-62c21deb4556" (UID: "d1d3f8b4-b1ba-4d61-b477-62c21deb4556"). InnerVolumeSpecName "kube-api-access-4frcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:58:37.183225 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.183206    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:37.183511 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.183500    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d3f8b4-b1ba-4d61-b477-62c21deb4556" containerName="test-curl-7305-140502673976608"
Aug 03 16:58:37.183553 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.183514    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d3f8b4-b1ba-4d61-b477-62c21deb4556" containerName="test-curl-7305-140502673976608"
Aug 03 16:58:37.183592 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.183572    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d3f8b4-b1ba-4d61-b477-62c21deb4556" containerName="test-curl-7305-140502673976608"
Aug 03 16:58:37.186391 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.186377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:37.194458 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.194409    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:37.281106 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.281068    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4frcw\" (UniqueName: \"kubernetes.io/projected/d1d3f8b4-b1ba-4d61-b477-62c21deb4556-kube-api-access-4frcw\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:37.382392 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.382353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw5db\" (UniqueName: \"kubernetes.io/projected/24ac58ce-27c0-48ac-b45d-e14a825f33de-kube-api-access-hw5db\") pod \"test-curl-7305-140502673976608\" (UID: \"24ac58ce-27c0-48ac-b45d-e14a825f33de\") " pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:37.483585 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.483489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw5db\" (UniqueName: \"kubernetes.io/projected/24ac58ce-27c0-48ac-b45d-e14a825f33de-kube-api-access-hw5db\") pod \"test-curl-7305-140502673976608\" (UID: \"24ac58ce-27c0-48ac-b45d-e14a825f33de\") " pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:37.492949 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.492914    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw5db\" (UniqueName: \"kubernetes.io/projected/24ac58ce-27c0-48ac-b45d-e14a825f33de-kube-api-access-hw5db\") pod \"test-curl-7305-140502673976608\" (UID: \"24ac58ce-27c0-48ac-b45d-e14a825f33de\") " pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:37.497638 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.497621    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:37.626058 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.626002    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:37.628201 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:58:37.628178    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24ac58ce_27c0_48ac_b45d_e14a825f33de.slice/crio-ac8136551a1f37a57259a1e99b868a9feb82e755ed69f477b6aa7d7098b6298e WatchSource:0}: Error finding container ac8136551a1f37a57259a1e99b868a9feb82e755ed69f477b6aa7d7098b6298e: Status 404 returned error can't find the container with id ac8136551a1f37a57259a1e99b868a9feb82e755ed69f477b6aa7d7098b6298e
Aug 03 16:58:37.888227 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.888197    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:37.888227 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.888207    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5aa44693d0fb47f993b5f7310489df7ac904c493270284726510359fca61bc45"
Aug 03 16:58:37.889406 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.889380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673976608" event={"ID":"24ac58ce-27c0-48ac-b45d-e14a825f33de","Type":"ContainerStarted","Data":"ac8136551a1f37a57259a1e99b868a9feb82e755ed69f477b6aa7d7098b6298e"}
Aug 03 16:58:37.900688 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:37.900649    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7305-140502673976608" oldPodUID="d1d3f8b4-b1ba-4d61-b477-62c21deb4556" podUID="24ac58ce-27c0-48ac-b45d-e14a825f33de"
Aug 03 16:58:38.243591 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:38.243514    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d3f8b4-b1ba-4d61-b477-62c21deb4556" path="/var/lib/kubelet/pods/d1d3f8b4-b1ba-4d61-b477-62c21deb4556/volumes"
Aug 03 16:58:38.895418 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:38.895384    2564 generic.go:356] "Generic (PLEG): container finished" podID="24ac58ce-27c0-48ac-b45d-e14a825f33de" containerID="9ad4d9959d0450f1d7106c9d669425c25de3603ee10e96dcb14b50ed9262e98d" exitCode=0
Aug 03 16:58:38.895592 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:38.895446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673976608" event={"ID":"24ac58ce-27c0-48ac-b45d-e14a825f33de","Type":"ContainerDied","Data":"9ad4d9959d0450f1d7106c9d669425c25de3603ee10e96dcb14b50ed9262e98d"}
Aug 03 16:58:40.033515 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.033490    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:40.056041 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.056012    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7305-140502673976608_24ac58ce-27c0-48ac-b45d-e14a825f33de/test-curl-7305-140502673976608/0.log"
Aug 03 16:58:40.080220 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.080194    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:40.084777 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.084755    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7305-140502673976608"]
Aug 03 16:58:40.101532 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.101511    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw5db\" (UniqueName: \"kubernetes.io/projected/24ac58ce-27c0-48ac-b45d-e14a825f33de-kube-api-access-hw5db\") pod \"24ac58ce-27c0-48ac-b45d-e14a825f33de\" (UID: \"24ac58ce-27c0-48ac-b45d-e14a825f33de\") "
Aug 03 16:58:40.103491 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.103465    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ac58ce-27c0-48ac-b45d-e14a825f33de-kube-api-access-hw5db" (OuterVolumeSpecName: "kube-api-access-hw5db") pod "24ac58ce-27c0-48ac-b45d-e14a825f33de" (UID: "24ac58ce-27c0-48ac-b45d-e14a825f33de"). InnerVolumeSpecName "kube-api-access-hw5db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:58:40.202219 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.202147    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw5db\" (UniqueName: \"kubernetes.io/projected/24ac58ce-27c0-48ac-b45d-e14a825f33de-kube-api-access-hw5db\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:40.235952 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.235924    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:40.236278 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.236265    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24ac58ce-27c0-48ac-b45d-e14a825f33de" containerName="test-curl-7305-140502673976608"
Aug 03 16:58:40.236321 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.236279    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ac58ce-27c0-48ac-b45d-e14a825f33de" containerName="test-curl-7305-140502673976608"
Aug 03 16:58:40.236356 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.236329    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="24ac58ce-27c0-48ac-b45d-e14a825f33de" containerName="test-curl-7305-140502673976608"
Aug 03 16:58:40.239132 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.239110    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:40.245010 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.244981    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24ac58ce-27c0-48ac-b45d-e14a825f33de" path="/var/lib/kubelet/pods/24ac58ce-27c0-48ac-b45d-e14a825f33de/volumes"
Aug 03 16:58:40.248170 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.248147    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:40.303116 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.303084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pwrx\" (UniqueName: \"kubernetes.io/projected/bd567a3d-d5ce-4f9a-b0eb-e57372e3beac-kube-api-access-2pwrx\") pod \"test-curl-7305-140502673932624\" (UID: \"bd567a3d-d5ce-4f9a-b0eb-e57372e3beac\") " pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:40.403782 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.403749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwrx\" (UniqueName: \"kubernetes.io/projected/bd567a3d-d5ce-4f9a-b0eb-e57372e3beac-kube-api-access-2pwrx\") pod \"test-curl-7305-140502673932624\" (UID: \"bd567a3d-d5ce-4f9a-b0eb-e57372e3beac\") " pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:40.413152 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.413129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pwrx\" (UniqueName: \"kubernetes.io/projected/bd567a3d-d5ce-4f9a-b0eb-e57372e3beac-kube-api-access-2pwrx\") pod \"test-curl-7305-140502673932624\" (UID: \"bd567a3d-d5ce-4f9a-b0eb-e57372e3beac\") " pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:40.549121 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.549033    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:40.673426 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.673398    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:40.675418 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:58:40.675387    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd567a3d_d5ce_4f9a_b0eb_e57372e3beac.slice/crio-6b8e08745a8af409f9895928794aaf4b85bfd56fa67cc4270f97056412c4b517 WatchSource:0}: Error finding container 6b8e08745a8af409f9895928794aaf4b85bfd56fa67cc4270f97056412c4b517: Status 404 returned error can't find the container with id 6b8e08745a8af409f9895928794aaf4b85bfd56fa67cc4270f97056412c4b517
Aug 03 16:58:40.904996 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.904967    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673976608"
Aug 03 16:58:40.904996 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.904994    2564 scope.go:117] "RemoveContainer" containerID="9ad4d9959d0450f1d7106c9d669425c25de3603ee10e96dcb14b50ed9262e98d"
Aug 03 16:58:40.906319 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:40.906117    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673932624" event={"ID":"bd567a3d-d5ce-4f9a-b0eb-e57372e3beac","Type":"ContainerStarted","Data":"6b8e08745a8af409f9895928794aaf4b85bfd56fa67cc4270f97056412c4b517"}
Aug 03 16:58:41.765496 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:41.765441    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw" podUID="fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" containerName="main" containerID="cri-o://80d32711347b5e632525a8e737d915c21d4945efd6939a1ca308af2b42071909" gracePeriod=45
Aug 03 16:58:41.913302 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:41.913268    2564 generic.go:356] "Generic (PLEG): container finished" podID="fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" containerID="80d32711347b5e632525a8e737d915c21d4945efd6939a1ca308af2b42071909" exitCode=0
Aug 03 16:58:41.913511 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:41.913343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw" event={"ID":"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5","Type":"ContainerDied","Data":"80d32711347b5e632525a8e737d915c21d4945efd6939a1ca308af2b42071909"}
Aug 03 16:58:41.917921 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:41.917796    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd567a3d-d5ce-4f9a-b0eb-e57372e3beac" containerID="1a66c4066d6646451adf6abdb258e7fba27ff83130b70948f68392856be648b9" exitCode=0
Aug 03 16:58:41.917921 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:41.917830    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673932624" event={"ID":"bd567a3d-d5ce-4f9a-b0eb-e57372e3beac","Type":"ContainerDied","Data":"1a66c4066d6646451adf6abdb258e7fba27ff83130b70948f68392856be648b9"}
Aug 03 16:58:42.008667 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.008645    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:58:42.017848 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.017789    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-model-cache\") pod \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") "
Aug 03 16:58:42.017848 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.017844    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-dshm\") pod \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") "
Aug 03 16:58:42.018026 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.017861    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tls-certs\") pod \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") "
Aug 03 16:58:42.018026 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.017881    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6fpw\" (UniqueName: \"kubernetes.io/projected/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-kube-api-access-n6fpw\") pod \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") "
Aug 03 16:58:42.018026 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.017902    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tmp-dir\") pod \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") "
Aug 03 16:58:42.018026 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.017931    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-home\") pod \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\" (UID: \"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5\") "
Aug 03 16:58:42.018210 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.018047    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" (UID: "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:58:42.018273 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.018212    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" (UID: "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:58:42.018273 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.018220    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:42.018356 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.018257    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-home" (OuterVolumeSpecName: "home") pod "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" (UID: "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:58:42.020109 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.020088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-dshm" (OuterVolumeSpecName: "dshm") pod "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" (UID: "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:58:42.020208 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.020146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-kube-api-access-n6fpw" (OuterVolumeSpecName: "kube-api-access-n6fpw") pod "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" (UID: "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5"). InnerVolumeSpecName "kube-api-access-n6fpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:58:42.020208 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.020161    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" (UID: "fb35f969-ad34-49f9-abe5-e40a1d1b1fe5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:58:42.118765 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.118714    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:42.118765 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.118759    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:42.118765 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.118770    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6fpw\" (UniqueName: \"kubernetes.io/projected/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-kube-api-access-n6fpw\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:42.118765 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.118779    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:42.119034 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.118787    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:42.924346 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.924315    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"
Aug 03 16:58:42.924816 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.924358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw" event={"ID":"fb35f969-ad34-49f9-abe5-e40a1d1b1fe5","Type":"ContainerDied","Data":"2fb3ab274e00730b032efc248a2d3bf143ed3cbf6d87494d71743b4df59f3222"}
Aug 03 16:58:42.924816 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.924396    2564 scope.go:117] "RemoveContainer" containerID="80d32711347b5e632525a8e737d915c21d4945efd6939a1ca308af2b42071909"
Aug 03 16:58:42.967983 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.967952    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"]
Aug 03 16:58:42.972627 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:42.972601    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/rate-test-model-7055f0-kserve-5576dc966c-4ptjw"]
Aug 03 16:58:43.055394 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.055370    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:43.077780 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.077762    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7305-140502673932624_bd567a3d-d5ce-4f9a-b0eb-e57372e3beac/test-curl-7305-140502673932624/0.log"
Aug 03 16:58:43.104856 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.104827    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:43.111201 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.111173    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:43.124952 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.124923    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pwrx\" (UniqueName: \"kubernetes.io/projected/bd567a3d-d5ce-4f9a-b0eb-e57372e3beac-kube-api-access-2pwrx\") pod \"bd567a3d-d5ce-4f9a-b0eb-e57372e3beac\" (UID: \"bd567a3d-d5ce-4f9a-b0eb-e57372e3beac\") "
Aug 03 16:58:43.127190 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.127166    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd567a3d-d5ce-4f9a-b0eb-e57372e3beac-kube-api-access-2pwrx" (OuterVolumeSpecName: "kube-api-access-2pwrx") pod "bd567a3d-d5ce-4f9a-b0eb-e57372e3beac" (UID: "bd567a3d-d5ce-4f9a-b0eb-e57372e3beac"). InnerVolumeSpecName "kube-api-access-2pwrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:58:43.226394 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.226329    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pwrx\" (UniqueName: \"kubernetes.io/projected/bd567a3d-d5ce-4f9a-b0eb-e57372e3beac-kube-api-access-2pwrx\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:43.231770 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.231744    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:43.232068 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.232056    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd567a3d-d5ce-4f9a-b0eb-e57372e3beac" containerName="test-curl-7305-140502673932624"
Aug 03 16:58:43.232112 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.232070    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd567a3d-d5ce-4f9a-b0eb-e57372e3beac" containerName="test-curl-7305-140502673932624"
Aug 03 16:58:43.232112 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.232086    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" containerName="main"
Aug 03 16:58:43.232112 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.232092    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" containerName="main"
Aug 03 16:58:43.232208 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.232145    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" containerName="main"
Aug 03 16:58:43.232208 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.232154    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd567a3d-d5ce-4f9a-b0eb-e57372e3beac" containerName="test-curl-7305-140502673932624"
Aug 03 16:58:43.235038 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.235023    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:43.239983 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.239957    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:43.327146 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.327108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trcxc\" (UniqueName: \"kubernetes.io/projected/34fdd244-6feb-4129-925d-6a5a2eeda48d-kube-api-access-trcxc\") pod \"test-curl-7305-140502673932624\" (UID: \"34fdd244-6feb-4129-925d-6a5a2eeda48d\") " pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:43.427920 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.427883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trcxc\" (UniqueName: \"kubernetes.io/projected/34fdd244-6feb-4129-925d-6a5a2eeda48d-kube-api-access-trcxc\") pod \"test-curl-7305-140502673932624\" (UID: \"34fdd244-6feb-4129-925d-6a5a2eeda48d\") " pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:43.438155 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.438124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trcxc\" (UniqueName: \"kubernetes.io/projected/34fdd244-6feb-4129-925d-6a5a2eeda48d-kube-api-access-trcxc\") pod \"test-curl-7305-140502673932624\" (UID: \"34fdd244-6feb-4129-925d-6a5a2eeda48d\") " pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:43.545303 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.545226    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:43.675021 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.674995    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:43.676614 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:58:43.676584    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34fdd244_6feb_4129_925d_6a5a2eeda48d.slice/crio-1fd7a4bb3dd7c36c71924ba61c9d8185aaae36e0ff66c2c64554839dfddabd64 WatchSource:0}: Error finding container 1fd7a4bb3dd7c36c71924ba61c9d8185aaae36e0ff66c2c64554839dfddabd64: Status 404 returned error can't find the container with id 1fd7a4bb3dd7c36c71924ba61c9d8185aaae36e0ff66c2c64554839dfddabd64
Aug 03 16:58:43.930329 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.930296    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673932624" event={"ID":"34fdd244-6feb-4129-925d-6a5a2eeda48d","Type":"ContainerStarted","Data":"1fd7a4bb3dd7c36c71924ba61c9d8185aaae36e0ff66c2c64554839dfddabd64"}
Aug 03 16:58:43.932371 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.932352    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b8e08745a8af409f9895928794aaf4b85bfd56fa67cc4270f97056412c4b517"
Aug 03 16:58:43.932454 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.932393    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:43.944877 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:43.944850    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7305-140502673932624" oldPodUID="bd567a3d-d5ce-4f9a-b0eb-e57372e3beac" podUID="34fdd244-6feb-4129-925d-6a5a2eeda48d"
Aug 03 16:58:44.243945 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:44.243856    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd567a3d-d5ce-4f9a-b0eb-e57372e3beac" path="/var/lib/kubelet/pods/bd567a3d-d5ce-4f9a-b0eb-e57372e3beac/volumes"
Aug 03 16:58:44.244312 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:44.244291    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb35f969-ad34-49f9-abe5-e40a1d1b1fe5" path="/var/lib/kubelet/pods/fb35f969-ad34-49f9-abe5-e40a1d1b1fe5/volumes"
Aug 03 16:58:44.938210 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:44.938175    2564 generic.go:356] "Generic (PLEG): container finished" podID="34fdd244-6feb-4129-925d-6a5a2eeda48d" containerID="7b6f02386efcc7097846a7cb834718a81eef79e209fdd350a928b426e88b8ed4" exitCode=0
Aug 03 16:58:44.938611 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:44.938259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140502673932624" event={"ID":"34fdd244-6feb-4129-925d-6a5a2eeda48d","Type":"ContainerDied","Data":"7b6f02386efcc7097846a7cb834718a81eef79e209fdd350a928b426e88b8ed4"}
Aug 03 16:58:46.070428 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.070396    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:46.093064 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.093035    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7305-140502673932624_34fdd244-6feb-4129-925d-6a5a2eeda48d/test-curl-7305-140502673932624/0.log"
Aug 03 16:58:46.119225 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.119183    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:46.123874 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.123841    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7305-140502673932624"]
Aug 03 16:58:46.151687 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.151653    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trcxc\" (UniqueName: \"kubernetes.io/projected/34fdd244-6feb-4129-925d-6a5a2eeda48d-kube-api-access-trcxc\") pod \"34fdd244-6feb-4129-925d-6a5a2eeda48d\" (UID: \"34fdd244-6feb-4129-925d-6a5a2eeda48d\") "
Aug 03 16:58:46.153785 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.153753    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34fdd244-6feb-4129-925d-6a5a2eeda48d-kube-api-access-trcxc" (OuterVolumeSpecName: "kube-api-access-trcxc") pod "34fdd244-6feb-4129-925d-6a5a2eeda48d" (UID: "34fdd244-6feb-4129-925d-6a5a2eeda48d"). InnerVolumeSpecName "kube-api-access-trcxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:58:46.243902 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.243825    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34fdd244-6feb-4129-925d-6a5a2eeda48d" path="/var/lib/kubelet/pods/34fdd244-6feb-4129-925d-6a5a2eeda48d/volumes"
Aug 03 16:58:46.253040 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.253006    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trcxc\" (UniqueName: \"kubernetes.io/projected/34fdd244-6feb-4129-925d-6a5a2eeda48d-kube-api-access-trcxc\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 16:58:46.948980 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.948952    2564 scope.go:117] "RemoveContainer" containerID="7b6f02386efcc7097846a7cb834718a81eef79e209fdd350a928b426e88b8ed4"
Aug 03 16:58:46.949148 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:46.948949    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140502673932624"
Aug 03 16:58:53.794449 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:58:53.794416    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:59:18.358324 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:18.358294    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:59:18.366929 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:18.366906    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 16:59:20.942635 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.942598    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"]
Aug 03 16:59:20.943071 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.942934    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34fdd244-6feb-4129-925d-6a5a2eeda48d" containerName="test-curl-7305-140502673932624"
Aug 03 16:59:20.943071 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.942946    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="34fdd244-6feb-4129-925d-6a5a2eeda48d" containerName="test-curl-7305-140502673932624"
Aug 03 16:59:20.943071 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.943002    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="34fdd244-6feb-4129-925d-6a5a2eeda48d" containerName="test-curl-7305-140502673932624"
Aug 03 16:59:20.947927 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.947907    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:20.950461 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.950429    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-cf8ed9-dockercfg-v55pr\""
Aug 03 16:59:20.950667 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.950645    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-cf8ed9\""
Aug 03 16:59:20.954602 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:20.954578    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"]
Aug 03 16:59:21.068948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.068912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbt5c\" (UniqueName: \"kubernetes.io/projected/f97c998f-e118-4fc7-8817-b78aa0fd28d2-kube-api-access-pbt5c\") pod \"payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:21.069134 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.068964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f97c998f-e118-4fc7-8817-b78aa0fd28d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:21.170085 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.170038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f97c998f-e118-4fc7-8817-b78aa0fd28d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:21.170288 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.170167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbt5c\" (UniqueName: \"kubernetes.io/projected/f97c998f-e118-4fc7-8817-b78aa0fd28d2-kube-api-access-pbt5c\") pod \"payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:21.170767 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.170719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f97c998f-e118-4fc7-8817-b78aa0fd28d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:21.179131 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.179112    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbt5c\" (UniqueName: \"kubernetes.io/projected/f97c998f-e118-4fc7-8817-b78aa0fd28d2-kube-api-access-pbt5c\") pod \"payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:21.258860 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.258763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:21.387640 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:21.387523    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"]
Aug 03 16:59:21.390001 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:59:21.389968    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf97c998f_e118_4fc7_8817_b78aa0fd28d2.slice/crio-102de57a41765ca4b41ad848281f35bfadf1ade662172e3c51a0cfe31809da2e WatchSource:0}: Error finding container 102de57a41765ca4b41ad848281f35bfadf1ade662172e3c51a0cfe31809da2e: Status 404 returned error can't find the container with id 102de57a41765ca4b41ad848281f35bfadf1ade662172e3c51a0cfe31809da2e
Aug 03 16:59:22.107000 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:22.106954    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5" event={"ID":"f97c998f-e118-4fc7-8817-b78aa0fd28d2","Type":"ContainerStarted","Data":"102de57a41765ca4b41ad848281f35bfadf1ade662172e3c51a0cfe31809da2e"}
Aug 03 16:59:23.195943 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:23.195908    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 16:59:25.121948 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:25.121909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5" event={"ID":"f97c998f-e118-4fc7-8817-b78aa0fd28d2","Type":"ContainerStarted","Data":"6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2"}
Aug 03 16:59:25.122405 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:25.122059    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:25.141151 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:25.141099    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5" podStartSLOduration=2.431987694 podStartE2EDuration="5.141082763s" totalImagesPullingTime="2.709095069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:59:25.139927657 +0000 UTC m=+3307.406743664" watchObservedRunningTime="2026-08-03 16:59:25.141082763 +0000 UTC m=+3307.407898792"
Aug 03 16:59:36.128599 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:36.128515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 16:59:51.287498 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:51.287421    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5_f97c998f-e118-4fc7-8817-b78aa0fd28d2/payload-processing/0.log"
Aug 03 16:59:56.954807 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:56.954769    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"]
Aug 03 16:59:56.961244 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:56.961223    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:56.964068 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:56.964050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0e4b78\"/\"ipp-route-0e4b78-kserve-self-signed-certs\""
Aug 03 16:59:56.965269 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:56.965249    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0e4b78\"/\"openshift-service-ca.crt\""
Aug 03 16:59:56.965372 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:56.965258    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0e4b78\"/\"default-dockercfg-w4fsd\""
Aug 03 16:59:56.965572 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:56.965555    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0e4b78\"/\"kube-root-ca.crt\""
Aug 03 16:59:56.971357 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:56.971335    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"]
Aug 03 16:59:57.085513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.085471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-home\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.085513 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.085510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-dshm\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.085727 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.085526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-tmp-dir\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.085727 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.085551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d905bf56-bae4-4e95-99ec-ce531a859407-tls-certs\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.085727 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.085617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvsqf\" (UniqueName: \"kubernetes.io/projected/d905bf56-bae4-4e95-99ec-ce531a859407-kube-api-access-gvsqf\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.085727 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.085679    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-model-cache\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.186607 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.186575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-model-cache\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.186780 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.186653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-home\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.186780 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.186673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-dshm\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.186780 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.186689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-tmp-dir\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.186780 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.186705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d905bf56-bae4-4e95-99ec-ce531a859407-tls-certs\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.186780 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.186724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvsqf\" (UniqueName: \"kubernetes.io/projected/d905bf56-bae4-4e95-99ec-ce531a859407-kube-api-access-gvsqf\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.187096 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.187069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-model-cache\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.187157 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.187124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-home\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.187208 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.187171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-tmp-dir\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.189017 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.188996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-dshm\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.189178 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.189162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d905bf56-bae4-4e95-99ec-ce531a859407-tls-certs\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.197986 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.197968    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvsqf\" (UniqueName: \"kubernetes.io/projected/d905bf56-bae4-4e95-99ec-ce531a859407-kube-api-access-gvsqf\") pod \"ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") " pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.271710 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.271629    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 16:59:57.404291 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:57.404262    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"]
Aug 03 16:59:57.407513 ip-10-0-138-110 kubenswrapper[2564]: W0803 16:59:57.407483    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd905bf56_bae4_4e95_99ec_ce531a859407.slice/crio-2ab98a0579ae39989ffce9651d24e94080a05bc0eb0715952a777b0bac91ce3b WatchSource:0}: Error finding container 2ab98a0579ae39989ffce9651d24e94080a05bc0eb0715952a777b0bac91ce3b: Status 404 returned error can't find the container with id 2ab98a0579ae39989ffce9651d24e94080a05bc0eb0715952a777b0bac91ce3b
Aug 03 16:59:58.267515 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:58.267479    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9" event={"ID":"d905bf56-bae4-4e95-99ec-ce531a859407","Type":"ContainerStarted","Data":"6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683"}
Aug 03 16:59:58.267515 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:58.267517    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9" event={"ID":"d905bf56-bae4-4e95-99ec-ce531a859407","Type":"ContainerStarted","Data":"2ab98a0579ae39989ffce9651d24e94080a05bc0eb0715952a777b0bac91ce3b"}
Aug 03 16:59:58.285446 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:58.285399    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9" podStartSLOduration=2.285384176 podStartE2EDuration="2.285384176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:59:58.284614366 +0000 UTC m=+3340.551430374" watchObservedRunningTime="2026-08-03 16:59:58.285384176 +0000 UTC m=+3340.552200180"
Aug 03 16:59:59.407884 ip-10-0-138-110 kubenswrapper[2564]: I0803 16:59:59.407852    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:00:00.140841 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.140804    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"]
Aug 03 17:00:00.144283 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.144260    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"]
Aug 03 17:00:00.144402 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.144367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"
Aug 03 17:00:00.147165 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.147143    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 17:00:00.147525 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.147511    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"]
Aug 03 17:00:00.147643 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.147628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"
Aug 03 17:00:00.150350 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.150330    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4"]
Aug 03 17:00:00.150460 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.150443    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"
Aug 03 17:00:00.153617 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.153599    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"]
Aug 03 17:00:00.153746 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.153711    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4"
Aug 03 17:00:00.158106 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.158005    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"]
Aug 03 17:00:00.158106 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.158085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"
Aug 03 17:00:00.158716 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.158694    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"
Aug 03 17:00:00.159790 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.159762    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"]
Aug 03 17:00:00.164730 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.162139    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"]
Aug 03 17:00:00.164730 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.164511    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4"]
Aug 03 17:00:00.168392 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.167001    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"]
Aug 03 17:00:00.171398 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.171380    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"
Aug 03 17:00:00.212875 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.212851    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"
Aug 03 17:00:00.221543 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.221230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4"
Aug 03 17:00:00.249523 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.249330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"
Aug 03 17:00:00.333750 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.333669    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"]
Aug 03 17:00:00.340324 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:00:00.340288    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7213c82_3043_429a_8bf9_641c5bae42c3.slice/crio-44fa2df1e1e3a6e9760282267d35e114a1f5cad24ef3158080c5c40a322bfb77 WatchSource:0}: Error finding container 44fa2df1e1e3a6e9760282267d35e114a1f5cad24ef3158080c5c40a322bfb77: Status 404 returned error can't find the container with id 44fa2df1e1e3a6e9760282267d35e114a1f5cad24ef3158080c5c40a322bfb77
Aug 03 17:00:00.449291 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.449265    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"]
Aug 03 17:00:00.451385 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:00:00.451358    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868f0af5_ae92_490b_ab80_38ae69445c92.slice/crio-843009cd81efac505fd4934bfd604b3b43ed9fb5cfe2108b2fecec1c5015c09a WatchSource:0}: Error finding container 843009cd81efac505fd4934bfd604b3b43ed9fb5cfe2108b2fecec1c5015c09a: Status 404 returned error can't find the container with id 843009cd81efac505fd4934bfd604b3b43ed9fb5cfe2108b2fecec1c5015c09a
Aug 03 17:00:00.575299 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.575279    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"]
Aug 03 17:00:00.603749 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:00:00.603711    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1afe3b3_216e_4c93_93e5_5d382c7fe832.slice/crio-637200ac0b9581064910aae87a0ac8a6fc8a7ea2d5f8f74d94da22679edeffc9 WatchSource:0}: Error finding container 637200ac0b9581064910aae87a0ac8a6fc8a7ea2d5f8f74d94da22679edeffc9: Status 404 returned error can't find the container with id 637200ac0b9581064910aae87a0ac8a6fc8a7ea2d5f8f74d94da22679edeffc9
Aug 03 17:00:00.618178 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.618157    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"]
Aug 03 17:00:00.623065 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:00:00.623037    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d28d34_45c2_4d2c_8f1e_27c7bd226896.slice/crio-6c42290c5df76886146ae7eec4dfe73d60b4b712bc4f1f18e4de24edc00596d9 WatchSource:0}: Error finding container 6c42290c5df76886146ae7eec4dfe73d60b4b712bc4f1f18e4de24edc00596d9: Status 404 returned error can't find the container with id 6c42290c5df76886146ae7eec4dfe73d60b4b712bc4f1f18e4de24edc00596d9
Aug 03 17:00:00.639416 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:00.639390    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4"]
Aug 03 17:00:00.645968 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:00:00.645941    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589e4a24_c479_49f4_a137_ff7e8eb6e302.slice/crio-191b0f5a2c5225547fc7f50d1fdda8ac50b452bf2efcc289f2754969ad175adc WatchSource:0}: Error finding container 191b0f5a2c5225547fc7f50d1fdda8ac50b452bf2efcc289f2754969ad175adc: Status 404 returned error can't find the container with id 191b0f5a2c5225547fc7f50d1fdda8ac50b452bf2efcc289f2754969ad175adc
Aug 03 17:00:01.289797 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.289730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt" event={"ID":"868f0af5-ae92-490b-ab80-38ae69445c92","Type":"ContainerStarted","Data":"f31e82a9795f0527e2a9ee17c3c41933870be8912fc524d7525df4ed39f90057"}
Aug 03 17:00:01.289982 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.289821    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt" event={"ID":"868f0af5-ae92-490b-ab80-38ae69445c92","Type":"ContainerStarted","Data":"843009cd81efac505fd4934bfd604b3b43ed9fb5cfe2108b2fecec1c5015c09a"}
Aug 03 17:00:01.291258 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.291232    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx" event={"ID":"a1afe3b3-216e-4c93-93e5-5d382c7fe832","Type":"ContainerStarted","Data":"f522dffc91bbbd60d63ba9d9b5dd9f8ef4bf9894f783d76a2d9b76a0f96656bb"}
Aug 03 17:00:01.291359 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.291280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx" event={"ID":"a1afe3b3-216e-4c93-93e5-5d382c7fe832","Type":"ContainerStarted","Data":"637200ac0b9581064910aae87a0ac8a6fc8a7ea2d5f8f74d94da22679edeffc9"}
Aug 03 17:00:01.292633 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.292613    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7" event={"ID":"a7213c82-3043-429a-8bf9-641c5bae42c3","Type":"ContainerStarted","Data":"805a53bf7c31090ee512b719776386d10eb1387f7daa4f721b51b6235bc224bc"}
Aug 03 17:00:01.292697 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.292651    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7" event={"ID":"a7213c82-3043-429a-8bf9-641c5bae42c3","Type":"ContainerStarted","Data":"44fa2df1e1e3a6e9760282267d35e114a1f5cad24ef3158080c5c40a322bfb77"}
Aug 03 17:00:01.294032 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.294003    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4" event={"ID":"589e4a24-c479-49f4-a137-ff7e8eb6e302","Type":"ContainerStarted","Data":"c10c26f12b58fad858fd483f603b9d5a33320ec679048beb6aac873f69f5e9e3"}
Aug 03 17:00:01.294130 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.294054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4" event={"ID":"589e4a24-c479-49f4-a137-ff7e8eb6e302","Type":"ContainerStarted","Data":"191b0f5a2c5225547fc7f50d1fdda8ac50b452bf2efcc289f2754969ad175adc"}
Aug 03 17:00:01.295188 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.295166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz" event={"ID":"b3d28d34-45c2-4d2c-8f1e-27c7bd226896","Type":"ContainerStarted","Data":"cec09256445ceabb878b35e6341fdb176d7f2ab7687936028dd2cfa258f37169"}
Aug 03 17:00:01.295250 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.295204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz" event={"ID":"b3d28d34-45c2-4d2c-8f1e-27c7bd226896","Type":"ContainerStarted","Data":"6c42290c5df76886146ae7eec4dfe73d60b4b712bc4f1f18e4de24edc00596d9"}
Aug 03 17:00:01.307412 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.307371    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt" podStartSLOduration=1.3073561009999999 podStartE2EDuration="1.307356101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:01.305769027 +0000 UTC m=+3343.572585035" watchObservedRunningTime="2026-08-03 17:00:01.307356101 +0000 UTC m=+3343.574172107"
Aug 03 17:00:01.320411 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.320375    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7" podStartSLOduration=1.32036376 podStartE2EDuration="1.32036376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:01.320094707 +0000 UTC m=+3343.586910712" watchObservedRunningTime="2026-08-03 17:00:01.32036376 +0000 UTC m=+3343.587179768"
Aug 03 17:00:01.337911 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.337874    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx" podStartSLOduration=1.337861016 podStartE2EDuration="1.337861016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:01.337173255 +0000 UTC m=+3343.603989257" watchObservedRunningTime="2026-08-03 17:00:01.337861016 +0000 UTC m=+3343.604677055"
Aug 03 17:00:01.354155 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.354109    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz" podStartSLOduration=1.354094049 podStartE2EDuration="1.354094049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:01.353543362 +0000 UTC m=+3343.620359369" watchObservedRunningTime="2026-08-03 17:00:01.354094049 +0000 UTC m=+3343.620910056"
Aug 03 17:00:01.371220 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:01.371180    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4" podStartSLOduration=1.371168338 podStartE2EDuration="1.371168338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:01.368983191 +0000 UTC m=+3343.635799207" watchObservedRunningTime="2026-08-03 17:00:01.371168338 +0000 UTC m=+3343.637984390"
Aug 03 17:00:02.301927 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.301885    2564 generic.go:356] "Generic (PLEG): container finished" podID="868f0af5-ae92-490b-ab80-38ae69445c92" containerID="f31e82a9795f0527e2a9ee17c3c41933870be8912fc524d7525df4ed39f90057" exitCode=0
Aug 03 17:00:02.302320 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.301975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt" event={"ID":"868f0af5-ae92-490b-ab80-38ae69445c92","Type":"ContainerDied","Data":"f31e82a9795f0527e2a9ee17c3c41933870be8912fc524d7525df4ed39f90057"}
Aug 03 17:00:02.303306 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.303285    2564 generic.go:356] "Generic (PLEG): container finished" podID="a1afe3b3-216e-4c93-93e5-5d382c7fe832" containerID="f522dffc91bbbd60d63ba9d9b5dd9f8ef4bf9894f783d76a2d9b76a0f96656bb" exitCode=0
Aug 03 17:00:02.303416 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.303349    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx" event={"ID":"a1afe3b3-216e-4c93-93e5-5d382c7fe832","Type":"ContainerDied","Data":"f522dffc91bbbd60d63ba9d9b5dd9f8ef4bf9894f783d76a2d9b76a0f96656bb"}
Aug 03 17:00:02.304561 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.304540    2564 generic.go:356] "Generic (PLEG): container finished" podID="a7213c82-3043-429a-8bf9-641c5bae42c3" containerID="805a53bf7c31090ee512b719776386d10eb1387f7daa4f721b51b6235bc224bc" exitCode=0
Aug 03 17:00:02.304641 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.304605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7" event={"ID":"a7213c82-3043-429a-8bf9-641c5bae42c3","Type":"ContainerDied","Data":"805a53bf7c31090ee512b719776386d10eb1387f7daa4f721b51b6235bc224bc"}
Aug 03 17:00:02.305866 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.305848    2564 generic.go:356] "Generic (PLEG): container finished" podID="589e4a24-c479-49f4-a137-ff7e8eb6e302" containerID="c10c26f12b58fad858fd483f603b9d5a33320ec679048beb6aac873f69f5e9e3" exitCode=0
Aug 03 17:00:02.305962 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.305947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4" event={"ID":"589e4a24-c479-49f4-a137-ff7e8eb6e302","Type":"ContainerDied","Data":"c10c26f12b58fad858fd483f603b9d5a33320ec679048beb6aac873f69f5e9e3"}
Aug 03 17:00:02.307328 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.307307    2564 generic.go:356] "Generic (PLEG): container finished" podID="b3d28d34-45c2-4d2c-8f1e-27c7bd226896" containerID="cec09256445ceabb878b35e6341fdb176d7f2ab7687936028dd2cfa258f37169" exitCode=0
Aug 03 17:00:02.307403 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:02.307341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz" event={"ID":"b3d28d34-45c2-4d2c-8f1e-27c7bd226896","Type":"ContainerDied","Data":"cec09256445ceabb878b35e6341fdb176d7f2ab7687936028dd2cfa258f37169"}
Aug 03 17:00:03.450278 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:03.450249    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"
Aug 03 17:00:03.495850 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:03.495823    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"
Aug 03 17:00:03.537914 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:03.537889    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"
Aug 03 17:00:03.540942 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:03.540923    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4"
Aug 03 17:00:03.544205 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:03.544187    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"
Aug 03 17:00:04.317826 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.317799    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4"
Aug 03 17:00:04.317969 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.317838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762940-zqbc4" event={"ID":"589e4a24-c479-49f4-a137-ff7e8eb6e302","Type":"ContainerDied","Data":"191b0f5a2c5225547fc7f50d1fdda8ac50b452bf2efcc289f2754969ad175adc"}
Aug 03 17:00:04.317969 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.317893    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="191b0f5a2c5225547fc7f50d1fdda8ac50b452bf2efcc289f2754969ad175adc"
Aug 03 17:00:04.319082 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.319065    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"
Aug 03 17:00:04.319209 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.319095    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz" event={"ID":"b3d28d34-45c2-4d2c-8f1e-27c7bd226896","Type":"ContainerDied","Data":"6c42290c5df76886146ae7eec4dfe73d60b4b712bc4f1f18e4de24edc00596d9"}
Aug 03 17:00:04.319209 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.319132    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c42290c5df76886146ae7eec4dfe73d60b4b712bc4f1f18e4de24edc00596d9"
Aug 03 17:00:04.320568 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.320547    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"
Aug 03 17:00:04.320665 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.320548    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt" event={"ID":"868f0af5-ae92-490b-ab80-38ae69445c92","Type":"ContainerDied","Data":"843009cd81efac505fd4934bfd604b3b43ed9fb5cfe2108b2fecec1c5015c09a"}
Aug 03 17:00:04.320712 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.320684    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="843009cd81efac505fd4934bfd604b3b43ed9fb5cfe2108b2fecec1c5015c09a"
Aug 03 17:00:04.321977 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.321960    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"
Aug 03 17:00:04.322077 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.321963    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx" event={"ID":"a1afe3b3-216e-4c93-93e5-5d382c7fe832","Type":"ContainerDied","Data":"637200ac0b9581064910aae87a0ac8a6fc8a7ea2d5f8f74d94da22679edeffc9"}
Aug 03 17:00:04.322077 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.322064    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="637200ac0b9581064910aae87a0ac8a6fc8a7ea2d5f8f74d94da22679edeffc9"
Aug 03 17:00:04.323327 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.323309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7" event={"ID":"a7213c82-3043-429a-8bf9-641c5bae42c3","Type":"ContainerDied","Data":"44fa2df1e1e3a6e9760282267d35e114a1f5cad24ef3158080c5c40a322bfb77"}
Aug 03 17:00:04.323327 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.323320    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"
Aug 03 17:00:04.323447 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:04.323340    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44fa2df1e1e3a6e9760282267d35e114a1f5cad24ef3158080c5c40a322bfb77"
Aug 03 17:00:07.272375 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:07.272334    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 17:00:07.272913 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:07.272398    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 17:00:07.284865 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:07.284836    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 17:00:07.351194 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:07.351161    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 17:00:24.900932 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:24.900896    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:00:34.437658 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:34.437627    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5_f97c998f-e118-4fc7-8817-b78aa0fd28d2/payload-processing/0.log"
Aug 03 17:00:39.590345 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590308    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"]
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590623    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="868f0af5-ae92-490b-ab80-38ae69445c92" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590633    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="868f0af5-ae92-490b-ab80-38ae69445c92" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590642    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589e4a24-c479-49f4-a137-ff7e8eb6e302" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590647    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="589e4a24-c479-49f4-a137-ff7e8eb6e302" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590655    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7213c82-3043-429a-8bf9-641c5bae42c3" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590660    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7213c82-3043-429a-8bf9-641c5bae42c3" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590671    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3d28d34-45c2-4d2c-8f1e-27c7bd226896" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590676    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3d28d34-45c2-4d2c-8f1e-27c7bd226896" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590684    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1afe3b3-216e-4c93-93e5-5d382c7fe832" containerName="cleanup"
Aug 03 17:00:39.590710 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590690    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1afe3b3-216e-4c93-93e5-5d382c7fe832" containerName="cleanup"
Aug 03 17:00:39.591041 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590757    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="589e4a24-c479-49f4-a137-ff7e8eb6e302" containerName="cleanup"
Aug 03 17:00:39.591041 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590765    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="868f0af5-ae92-490b-ab80-38ae69445c92" containerName="cleanup"
Aug 03 17:00:39.591041 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590770    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1afe3b3-216e-4c93-93e5-5d382c7fe832" containerName="cleanup"
Aug 03 17:00:39.591041 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590777    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7213c82-3043-429a-8bf9-641c5bae42c3" containerName="cleanup"
Aug 03 17:00:39.591041 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.590784    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3d28d34-45c2-4d2c-8f1e-27c7bd226896" containerName="cleanup"
Aug 03 17:00:39.594004 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.593983    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.596975 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.596952    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4df7e0\""
Aug 03 17:00:39.596975 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.596952    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4df7e0-dockercfg-f52k9\""
Aug 03 17:00:39.603599 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.603577    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"]
Aug 03 17:00:39.635129 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.635104    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"]
Aug 03 17:00:39.638261 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.638245    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:39.647941 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.647921    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"]
Aug 03 17:00:39.681668 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.681646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k9zz\" (UniqueName: \"kubernetes.io/projected/53c8d310-2d11-47bb-bc52-77d48c8c4522-kube-api-access-7k9zz\") pod \"payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.681796 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.681675    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c8d310-2d11-47bb-bc52-77d48c8c4522-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.782487 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.782440    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k9zz\" (UniqueName: \"kubernetes.io/projected/53c8d310-2d11-47bb-bc52-77d48c8c4522-kube-api-access-7k9zz\") pod \"payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.782487 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.782493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c8d310-2d11-47bb-bc52-77d48c8c4522-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.782723 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.782593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6xnz\" (UniqueName: \"kubernetes.io/projected/e29e5620-6f43-4320-9dac-7a0ea50df794-kube-api-access-k6xnz\") pod \"payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:39.782723 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.782636    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29e5620-6f43-4320-9dac-7a0ea50df794-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:39.783218 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.783199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c8d310-2d11-47bb-bc52-77d48c8c4522-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.791897 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.791869    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k9zz\" (UniqueName: \"kubernetes.io/projected/53c8d310-2d11-47bb-bc52-77d48c8c4522-kube-api-access-7k9zz\") pod \"payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.883142 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.883054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xnz\" (UniqueName: \"kubernetes.io/projected/e29e5620-6f43-4320-9dac-7a0ea50df794-kube-api-access-k6xnz\") pod \"payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:39.883142 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.883109    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29e5620-6f43-4320-9dac-7a0ea50df794-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:39.883766 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.883748    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29e5620-6f43-4320-9dac-7a0ea50df794-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:39.892076 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.892048    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6xnz\" (UniqueName: \"kubernetes.io/projected/e29e5620-6f43-4320-9dac-7a0ea50df794-kube-api-access-k6xnz\") pod \"payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:39.904231 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.904210    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:39.948045 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:39.948014    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:40.053095 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.053066    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"]
Aug 03 17:00:40.055261 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:00:40.055234    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c8d310_2d11_47bb_bc52_77d48c8c4522.slice/crio-18a61c5219e3d9f174e3d8d5e42e7973315c5447c943be7f23f165577d3c7ad2 WatchSource:0}: Error finding container 18a61c5219e3d9f174e3d8d5e42e7973315c5447c943be7f23f165577d3c7ad2: Status 404 returned error can't find the container with id 18a61c5219e3d9f174e3d8d5e42e7973315c5447c943be7f23f165577d3c7ad2
Aug 03 17:00:40.106265 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.106240    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"]
Aug 03 17:00:40.108065 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:00:40.108038    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29e5620_6f43_4320_9dac_7a0ea50df794.slice/crio-73814a8e5761a4a180c4685fad463f6e025842caebf6df191218898408062407 WatchSource:0}: Error finding container 73814a8e5761a4a180c4685fad463f6e025842caebf6df191218898408062407: Status 404 returned error can't find the container with id 73814a8e5761a4a180c4685fad463f6e025842caebf6df191218898408062407
Aug 03 17:00:40.492672 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.492573    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl" event={"ID":"53c8d310-2d11-47bb-bc52-77d48c8c4522","Type":"ContainerStarted","Data":"625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2"}
Aug 03 17:00:40.492672 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.492635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl" event={"ID":"53c8d310-2d11-47bb-bc52-77d48c8c4522","Type":"ContainerStarted","Data":"18a61c5219e3d9f174e3d8d5e42e7973315c5447c943be7f23f165577d3c7ad2"}
Aug 03 17:00:40.492672 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.492664    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:00:40.493966 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.493940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv" event={"ID":"e29e5620-6f43-4320-9dac-7a0ea50df794","Type":"ContainerStarted","Data":"c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473"}
Aug 03 17:00:40.494085 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.493979    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv" event={"ID":"e29e5620-6f43-4320-9dac-7a0ea50df794","Type":"ContainerStarted","Data":"73814a8e5761a4a180c4685fad463f6e025842caebf6df191218898408062407"}
Aug 03 17:00:40.494085 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.494019    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:00:40.510960 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.510906    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl" podStartSLOduration=1.51088755 podStartE2EDuration="1.51088755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:40.508270825 +0000 UTC m=+3382.775086930" watchObservedRunningTime="2026-08-03 17:00:40.51088755 +0000 UTC m=+3382.777703558"
Aug 03 17:00:40.524036 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:40.523988    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv" podStartSLOduration=1.523969874 podStartE2EDuration="1.523969874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:40.523532252 +0000 UTC m=+3382.790348255" watchObservedRunningTime="2026-08-03 17:00:40.523969874 +0000 UTC m=+3382.790785878"
Aug 03 17:00:43.389141 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:43.389104    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:00:51.500821 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:51.500781    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl" podUID="53c8d310-2d11-47bb-bc52-77d48c8c4522" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.51:9004: connect: connection refused"
Aug 03 17:00:51.501322 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:00:51.501302    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:01:01.500643 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:01.500610    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:01:03.737666 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.737629    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"]
Aug 03 17:01:03.741106 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.741088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"
Aug 03 17:01:03.743930 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.743905    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 03 17:01:03.744061 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.743981    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 17:01:03.749481 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.749457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"]
Aug 03 17:01:03.786024 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.785996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6b728e-667e-40d8-873f-51eea575595c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc\" (UID: \"6e6b728e-667e-40d8-873f-51eea575595c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"
Aug 03 17:01:03.886949 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.886914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6b728e-667e-40d8-873f-51eea575595c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc\" (UID: \"6e6b728e-667e-40d8-873f-51eea575595c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"
Aug 03 17:01:03.887502 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:03.887483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6b728e-667e-40d8-873f-51eea575595c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc\" (UID: \"6e6b728e-667e-40d8-873f-51eea575595c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"
Aug 03 17:01:04.052602 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:04.052513    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"
Aug 03 17:01:04.202863 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:04.202838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"]
Aug 03 17:01:04.204633 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:01:04.204609    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6b728e_667e_40d8_873f_51eea575595c.slice/crio-49bab9a36307278ef9a25e9db8fd68eceb845357f04c8a7d15bc29332ebf64a1 WatchSource:0}: Error finding container 49bab9a36307278ef9a25e9db8fd68eceb845357f04c8a7d15bc29332ebf64a1: Status 404 returned error can't find the container with id 49bab9a36307278ef9a25e9db8fd68eceb845357f04c8a7d15bc29332ebf64a1
Aug 03 17:01:04.609810 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:04.609710    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc" event={"ID":"6e6b728e-667e-40d8-873f-51eea575595c","Type":"ContainerStarted","Data":"60d81412e30969290d63acaf63b04a028d54dc9fc5c5f61388248de30e1de031"}
Aug 03 17:01:04.609810 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:04.609780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc" event={"ID":"6e6b728e-667e-40d8-873f-51eea575595c","Type":"ContainerStarted","Data":"49bab9a36307278ef9a25e9db8fd68eceb845357f04c8a7d15bc29332ebf64a1"}
Aug 03 17:01:04.627084 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:04.627034    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc" podStartSLOduration=1.627017343 podStartE2EDuration="1.627017343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:01:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:01:04.625821958 +0000 UTC m=+3406.892637966" watchObservedRunningTime="2026-08-03 17:01:04.627017343 +0000 UTC m=+3406.893833351"
Aug 03 17:01:05.616410 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:05.616374    2564 generic.go:356] "Generic (PLEG): container finished" podID="6e6b728e-667e-40d8-873f-51eea575595c" containerID="60d81412e30969290d63acaf63b04a028d54dc9fc5c5f61388248de30e1de031" exitCode=0
Aug 03 17:01:05.616929 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:05.616460    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc" event={"ID":"6e6b728e-667e-40d8-873f-51eea575595c","Type":"ContainerDied","Data":"60d81412e30969290d63acaf63b04a028d54dc9fc5c5f61388248de30e1de031"}
Aug 03 17:01:06.751211 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:06.751179    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"
Aug 03 17:01:06.813585 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:06.813550    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6b728e-667e-40d8-873f-51eea575595c-maas-api-service-ca\") pod \"6e6b728e-667e-40d8-873f-51eea575595c\" (UID: \"6e6b728e-667e-40d8-873f-51eea575595c\") "
Aug 03 17:01:06.813979 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:06.813954    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e6b728e-667e-40d8-873f-51eea575595c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e6b728e-667e-40d8-873f-51eea575595c" (UID: "6e6b728e-667e-40d8-873f-51eea575595c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:06.915108 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:06.915021    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6b728e-667e-40d8-873f-51eea575595c-maas-api-service-ca\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:07.626895 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:07.626855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc" event={"ID":"6e6b728e-667e-40d8-873f-51eea575595c","Type":"ContainerDied","Data":"49bab9a36307278ef9a25e9db8fd68eceb845357f04c8a7d15bc29332ebf64a1"}
Aug 03 17:01:07.626895 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:07.626871    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"
Aug 03 17:01:07.626895 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:07.626905    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49bab9a36307278ef9a25e9db8fd68eceb845357f04c8a7d15bc29332ebf64a1"
Aug 03 17:01:09.606035 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.606002    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"]
Aug 03 17:01:09.606527 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.606207    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv" podUID="e29e5620-6f43-4320-9dac-7a0ea50df794" containerName="payload-processing" containerID="cri-o://c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473" gracePeriod=30
Aug 03 17:01:09.610789 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.610757    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"]
Aug 03 17:01:09.611017 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.610985    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl" podUID="53c8d310-2d11-47bb-bc52-77d48c8c4522" containerName="payload-pre-processing" containerID="cri-o://625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2" gracePeriod=30
Aug 03 17:01:09.747437 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.747374    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"]
Aug 03 17:01:09.750372 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.750345    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df7e0-fnndc"]
Aug 03 17:01:09.864633 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.864568    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:01:09.888487 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.888466    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:01:09.944564 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.944529    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6xnz\" (UniqueName: \"kubernetes.io/projected/e29e5620-6f43-4320-9dac-7a0ea50df794-kube-api-access-k6xnz\") pod \"e29e5620-6f43-4320-9dac-7a0ea50df794\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") "
Aug 03 17:01:09.944747 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.944621    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k9zz\" (UniqueName: \"kubernetes.io/projected/53c8d310-2d11-47bb-bc52-77d48c8c4522-kube-api-access-7k9zz\") pod \"53c8d310-2d11-47bb-bc52-77d48c8c4522\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") "
Aug 03 17:01:09.944747 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.944665    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29e5620-6f43-4320-9dac-7a0ea50df794-plugins-config-volume\") pod \"e29e5620-6f43-4320-9dac-7a0ea50df794\" (UID: \"e29e5620-6f43-4320-9dac-7a0ea50df794\") "
Aug 03 17:01:09.944747 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.944705    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c8d310-2d11-47bb-bc52-77d48c8c4522-plugins-config-volume\") pod \"53c8d310-2d11-47bb-bc52-77d48c8c4522\" (UID: \"53c8d310-2d11-47bb-bc52-77d48c8c4522\") "
Aug 03 17:01:09.945115 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.945089    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29e5620-6f43-4320-9dac-7a0ea50df794-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e29e5620-6f43-4320-9dac-7a0ea50df794" (UID: "e29e5620-6f43-4320-9dac-7a0ea50df794"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:09.945244 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.945115    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53c8d310-2d11-47bb-bc52-77d48c8c4522-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53c8d310-2d11-47bb-bc52-77d48c8c4522" (UID: "53c8d310-2d11-47bb-bc52-77d48c8c4522"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:09.946962 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.946939    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29e5620-6f43-4320-9dac-7a0ea50df794-kube-api-access-k6xnz" (OuterVolumeSpecName: "kube-api-access-k6xnz") pod "e29e5620-6f43-4320-9dac-7a0ea50df794" (UID: "e29e5620-6f43-4320-9dac-7a0ea50df794"). InnerVolumeSpecName "kube-api-access-k6xnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:01:09.947065 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:09.946969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c8d310-2d11-47bb-bc52-77d48c8c4522-kube-api-access-7k9zz" (OuterVolumeSpecName: "kube-api-access-7k9zz") pod "53c8d310-2d11-47bb-bc52-77d48c8c4522" (UID: "53c8d310-2d11-47bb-bc52-77d48c8c4522"). InnerVolumeSpecName "kube-api-access-7k9zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:01:10.046152 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.046063    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6xnz\" (UniqueName: \"kubernetes.io/projected/e29e5620-6f43-4320-9dac-7a0ea50df794-kube-api-access-k6xnz\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:10.046152 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.046098    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k9zz\" (UniqueName: \"kubernetes.io/projected/53c8d310-2d11-47bb-bc52-77d48c8c4522-kube-api-access-7k9zz\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:10.046152 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.046107    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29e5620-6f43-4320-9dac-7a0ea50df794-plugins-config-volume\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:10.046152 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.046116    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c8d310-2d11-47bb-bc52-77d48c8c4522-plugins-config-volume\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:10.248865 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.248788    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e6b728e-667e-40d8-873f-51eea575595c" path="/var/lib/kubelet/pods/6e6b728e-667e-40d8-873f-51eea575595c/volumes"
Aug 03 17:01:10.646553 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.646521    2564 generic.go:356] "Generic (PLEG): container finished" podID="53c8d310-2d11-47bb-bc52-77d48c8c4522" containerID="625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2" exitCode=0
Aug 03 17:01:10.647005 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.646580    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"
Aug 03 17:01:10.647005 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.646604    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl" event={"ID":"53c8d310-2d11-47bb-bc52-77d48c8c4522","Type":"ContainerDied","Data":"625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2"}
Aug 03 17:01:10.647005 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.646655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl" event={"ID":"53c8d310-2d11-47bb-bc52-77d48c8c4522","Type":"ContainerDied","Data":"18a61c5219e3d9f174e3d8d5e42e7973315c5447c943be7f23f165577d3c7ad2"}
Aug 03 17:01:10.647005 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.646677    2564 scope.go:117] "RemoveContainer" containerID="625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2"
Aug 03 17:01:10.648086 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.648049    2564 generic.go:356] "Generic (PLEG): container finished" podID="e29e5620-6f43-4320-9dac-7a0ea50df794" containerID="c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473" exitCode=0
Aug 03 17:01:10.648151 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.648086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv" event={"ID":"e29e5620-6f43-4320-9dac-7a0ea50df794","Type":"ContainerDied","Data":"c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473"}
Aug 03 17:01:10.648151 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.648109    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"
Aug 03 17:01:10.648151 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.648121    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv" event={"ID":"e29e5620-6f43-4320-9dac-7a0ea50df794","Type":"ContainerDied","Data":"73814a8e5761a4a180c4685fad463f6e025842caebf6df191218898408062407"}
Aug 03 17:01:10.655429 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.655413    2564 scope.go:117] "RemoveContainer" containerID="625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2"
Aug 03 17:01:10.655663 ip-10-0-138-110 kubenswrapper[2564]: E0803 17:01:10.655645    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2\": container with ID starting with 625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2 not found: ID does not exist" containerID="625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2"
Aug 03 17:01:10.655699 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.655671    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2"} err="failed to get container status \"625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2\": rpc error: code = NotFound desc = could not find container \"625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2\": container with ID starting with 625e37331d2947d69f56059ac3fc2c6d6a5324c33c5e514e77396d1d25ac9ce2 not found: ID does not exist"
Aug 03 17:01:10.655699 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.655688    2564 scope.go:117] "RemoveContainer" containerID="c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473"
Aug 03 17:01:10.662979 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.662961    2564 scope.go:117] "RemoveContainer" containerID="c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473"
Aug 03 17:01:10.663211 ip-10-0-138-110 kubenswrapper[2564]: E0803 17:01:10.663193    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473\": container with ID starting with c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473 not found: ID does not exist" containerID="c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473"
Aug 03 17:01:10.663269 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.663217    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473"} err="failed to get container status \"c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473\": rpc error: code = NotFound desc = could not find container \"c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473\": container with ID starting with c4c87218301263e3ae391778b1485d25ec276005e593748d289e121f47d76473 not found: ID does not exist"
Aug 03 17:01:10.666450 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.666427    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"]
Aug 03 17:01:10.670999 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.670979    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4df7e0-7777cb6d84-hfzpv"]
Aug 03 17:01:10.680989 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.680969    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"]
Aug 03 17:01:10.685055 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:10.685032    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4df7e0-557b67457b-kfxzl"]
Aug 03 17:01:12.243433 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:12.243398    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c8d310-2d11-47bb-bc52-77d48c8c4522" path="/var/lib/kubelet/pods/53c8d310-2d11-47bb-bc52-77d48c8c4522/volumes"
Aug 03 17:01:12.243843 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:12.243769    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e29e5620-6f43-4320-9dac-7a0ea50df794" path="/var/lib/kubelet/pods/e29e5620-6f43-4320-9dac-7a0ea50df794/volumes"
Aug 03 17:01:12.897753 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:12.897714    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:01:21.183920 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.183889    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"]
Aug 03 17:01:21.184302 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184203    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29e5620-6f43-4320-9dac-7a0ea50df794" containerName="payload-processing"
Aug 03 17:01:21.184302 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184213    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29e5620-6f43-4320-9dac-7a0ea50df794" containerName="payload-processing"
Aug 03 17:01:21.184302 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184225    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c8d310-2d11-47bb-bc52-77d48c8c4522" containerName="payload-pre-processing"
Aug 03 17:01:21.184302 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184231    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c8d310-2d11-47bb-bc52-77d48c8c4522" containerName="payload-pre-processing"
Aug 03 17:01:21.184302 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184246    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e6b728e-667e-40d8-873f-51eea575595c" containerName="revoke-keys"
Aug 03 17:01:21.184302 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184251    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e6b728e-667e-40d8-873f-51eea575595c" containerName="revoke-keys"
Aug 03 17:01:21.184302 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184306    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c8d310-2d11-47bb-bc52-77d48c8c4522" containerName="payload-pre-processing"
Aug 03 17:01:21.184526 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184314    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29e5620-6f43-4320-9dac-7a0ea50df794" containerName="payload-processing"
Aug 03 17:01:21.184526 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.184323    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e6b728e-667e-40d8-873f-51eea575595c" containerName="revoke-keys"
Aug 03 17:01:21.188940 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.188922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"
Aug 03 17:01:21.191842 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.191822    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 17:01:21.191946 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.191902    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 03 17:01:21.195799 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.195776    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"]
Aug 03 17:01:21.347164 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.347127    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80bec39f-fa49-4cf4-a916-9adab15f4504-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp\" (UID: \"80bec39f-fa49-4cf4-a916-9adab15f4504\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"
Aug 03 17:01:21.448229 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.448143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80bec39f-fa49-4cf4-a916-9adab15f4504-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp\" (UID: \"80bec39f-fa49-4cf4-a916-9adab15f4504\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"
Aug 03 17:01:21.448778 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.448759    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80bec39f-fa49-4cf4-a916-9adab15f4504-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp\" (UID: \"80bec39f-fa49-4cf4-a916-9adab15f4504\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"
Aug 03 17:01:21.501217 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.501179    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"
Aug 03 17:01:21.643844 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.643813    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"]
Aug 03 17:01:21.704524 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:21.704358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp" event={"ID":"80bec39f-fa49-4cf4-a916-9adab15f4504","Type":"ContainerStarted","Data":"3e57a9665a317f28afc66c8ef90bd7a395d2a3911ae1273d4e8cfaf20cc90549"}
Aug 03 17:01:22.710700 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:22.710665    2564 generic.go:356] "Generic (PLEG): container finished" podID="80bec39f-fa49-4cf4-a916-9adab15f4504" containerID="103fd0cb4c85ad210dfa1ac452c971f241ff48b7d16cd0f7560e92366a81e552" exitCode=0
Aug 03 17:01:22.711105 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:22.710773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp" event={"ID":"80bec39f-fa49-4cf4-a916-9adab15f4504","Type":"ContainerDied","Data":"103fd0cb4c85ad210dfa1ac452c971f241ff48b7d16cd0f7560e92366a81e552"}
Aug 03 17:01:23.844975 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:23.844952    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"
Aug 03 17:01:23.972579 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:23.972485    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80bec39f-fa49-4cf4-a916-9adab15f4504-maas-api-service-ca\") pod \"80bec39f-fa49-4cf4-a916-9adab15f4504\" (UID: \"80bec39f-fa49-4cf4-a916-9adab15f4504\") "
Aug 03 17:01:23.972878 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:23.972855    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80bec39f-fa49-4cf4-a916-9adab15f4504-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "80bec39f-fa49-4cf4-a916-9adab15f4504" (UID: "80bec39f-fa49-4cf4-a916-9adab15f4504"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:24.073327 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:24.073290    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80bec39f-fa49-4cf4-a916-9adab15f4504-maas-api-service-ca\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:24.722185 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:24.722149    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"
Aug 03 17:01:24.722185 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:24.722161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp" event={"ID":"80bec39f-fa49-4cf4-a916-9adab15f4504","Type":"ContainerDied","Data":"3e57a9665a317f28afc66c8ef90bd7a395d2a3911ae1273d4e8cfaf20cc90549"}
Aug 03 17:01:24.722448 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:24.722235    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e57a9665a317f28afc66c8ef90bd7a395d2a3911ae1273d4e8cfaf20cc90549"
Aug 03 17:01:34.396473 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:34.396436    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:01:36.648951 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:36.648914    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"]
Aug 03 17:01:36.651911 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:36.651883    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-0e4b78-29762940-lrqx7"]
Aug 03 17:01:36.824941 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:36.824902    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"]
Aug 03 17:01:36.827781 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:36.827754    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0e4b78-xz6rp"]
Aug 03 17:01:38.243546 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:38.243513    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80bec39f-fa49-4cf4-a916-9adab15f4504" path="/var/lib/kubelet/pods/80bec39f-fa49-4cf4-a916-9adab15f4504/volumes"
Aug 03 17:01:38.243951 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:38.243823    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7213c82-3043-429a-8bf9-641c5bae42c3" path="/var/lib/kubelet/pods/a7213c82-3043-429a-8bf9-641c5bae42c3/volumes"
Aug 03 17:01:42.331934 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:42.331901    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"]
Aug 03 17:01:42.689665 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:42.689578    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:01:57.350195 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.350144    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9" podUID="d905bf56-bae4-4e95-99ec-ce531a859407" containerName="main" containerID="cri-o://6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683" gracePeriod=45
Aug 03 17:01:57.605128 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.605071    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 17:01:57.669181 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669151    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-dshm\") pod \"d905bf56-bae4-4e95-99ec-ce531a859407\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") "
Aug 03 17:01:57.669340 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669267    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d905bf56-bae4-4e95-99ec-ce531a859407-tls-certs\") pod \"d905bf56-bae4-4e95-99ec-ce531a859407\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") "
Aug 03 17:01:57.669340 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669318    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvsqf\" (UniqueName: \"kubernetes.io/projected/d905bf56-bae4-4e95-99ec-ce531a859407-kube-api-access-gvsqf\") pod \"d905bf56-bae4-4e95-99ec-ce531a859407\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") "
Aug 03 17:01:57.669340 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669335    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-model-cache\") pod \"d905bf56-bae4-4e95-99ec-ce531a859407\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") "
Aug 03 17:01:57.669538 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669392    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-tmp-dir\") pod \"d905bf56-bae4-4e95-99ec-ce531a859407\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") "
Aug 03 17:01:57.669538 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669412    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-home\") pod \"d905bf56-bae4-4e95-99ec-ce531a859407\" (UID: \"d905bf56-bae4-4e95-99ec-ce531a859407\") "
Aug 03 17:01:57.669679 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669650    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-model-cache" (OuterVolumeSpecName: "model-cache") pod "d905bf56-bae4-4e95-99ec-ce531a859407" (UID: "d905bf56-bae4-4e95-99ec-ce531a859407"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 17:01:57.669679 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669663    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d905bf56-bae4-4e95-99ec-ce531a859407" (UID: "d905bf56-bae4-4e95-99ec-ce531a859407"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 17:01:57.669862 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.669845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-home" (OuterVolumeSpecName: "home") pod "d905bf56-bae4-4e95-99ec-ce531a859407" (UID: "d905bf56-bae4-4e95-99ec-ce531a859407"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 17:01:57.671431 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.671400    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-dshm" (OuterVolumeSpecName: "dshm") pod "d905bf56-bae4-4e95-99ec-ce531a859407" (UID: "d905bf56-bae4-4e95-99ec-ce531a859407"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 17:01:57.671580 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.671469    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d905bf56-bae4-4e95-99ec-ce531a859407-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d905bf56-bae4-4e95-99ec-ce531a859407" (UID: "d905bf56-bae4-4e95-99ec-ce531a859407"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 17:01:57.671672 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.671653    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d905bf56-bae4-4e95-99ec-ce531a859407-kube-api-access-gvsqf" (OuterVolumeSpecName: "kube-api-access-gvsqf") pod "d905bf56-bae4-4e95-99ec-ce531a859407" (UID: "d905bf56-bae4-4e95-99ec-ce531a859407"). InnerVolumeSpecName "kube-api-access-gvsqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:01:57.770453 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.770421    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-tmp-dir\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:57.770453 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.770448    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-home\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:57.770453 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.770457    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-dshm\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:57.770720 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.770465    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d905bf56-bae4-4e95-99ec-ce531a859407-tls-certs\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:57.770720 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.770479    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvsqf\" (UniqueName: \"kubernetes.io/projected/d905bf56-bae4-4e95-99ec-ce531a859407-kube-api-access-gvsqf\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:57.770720 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.770490    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d905bf56-bae4-4e95-99ec-ce531a859407-model-cache\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:01:57.886715 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.886629    2564 generic.go:356] "Generic (PLEG): container finished" podID="d905bf56-bae4-4e95-99ec-ce531a859407" containerID="6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683" exitCode=0
Aug 03 17:01:57.886715 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.886673    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"
Aug 03 17:01:57.886912 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.886714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9" event={"ID":"d905bf56-bae4-4e95-99ec-ce531a859407","Type":"ContainerDied","Data":"6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683"}
Aug 03 17:01:57.886912 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.886771    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9" event={"ID":"d905bf56-bae4-4e95-99ec-ce531a859407","Type":"ContainerDied","Data":"2ab98a0579ae39989ffce9651d24e94080a05bc0eb0715952a777b0bac91ce3b"}
Aug 03 17:01:57.886912 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.886789    2564 scope.go:117] "RemoveContainer" containerID="6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683"
Aug 03 17:01:57.895617 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.895599    2564 scope.go:117] "RemoveContainer" containerID="6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683"
Aug 03 17:01:57.895877 ip-10-0-138-110 kubenswrapper[2564]: E0803 17:01:57.895856    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683\": container with ID starting with 6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683 not found: ID does not exist" containerID="6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683"
Aug 03 17:01:57.895949 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.895885    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683"} err="failed to get container status \"6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683\": rpc error: code = NotFound desc = could not find container \"6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683\": container with ID starting with 6a2ff2565870234a945106edb916bd19d4a4e3db41f990b3dd881e323c8bf683 not found: ID does not exist"
Aug 03 17:01:57.910594 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.910569    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"]
Aug 03 17:01:57.915293 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:57.915274    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-0e4b78/ipp-route-0e4b78-kserve-7c5b6fcc95-jnqx9"]
Aug 03 17:01:58.243873 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:01:58.243792    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d905bf56-bae4-4e95-99ec-ce531a859407" path="/var/lib/kubelet/pods/d905bf56-bae4-4e95-99ec-ce531a859407/volumes"
Aug 03 17:02:41.912833 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.912745    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"]
Aug 03 17:02:41.913350 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.913212    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80bec39f-fa49-4cf4-a916-9adab15f4504" containerName="revoke-keys"
Aug 03 17:02:41.913350 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.913230    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="80bec39f-fa49-4cf4-a916-9adab15f4504" containerName="revoke-keys"
Aug 03 17:02:41.913350 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.913247    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d905bf56-bae4-4e95-99ec-ce531a859407" containerName="main"
Aug 03 17:02:41.913350 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.913255    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d905bf56-bae4-4e95-99ec-ce531a859407" containerName="main"
Aug 03 17:02:41.913567 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.913366    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="80bec39f-fa49-4cf4-a916-9adab15f4504" containerName="revoke-keys"
Aug 03 17:02:41.913567 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.913381    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d905bf56-bae4-4e95-99ec-ce531a859407" containerName="main"
Aug 03 17:02:41.916285 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.916265    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"
Aug 03 17:02:41.919708 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.919685    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 03 17:02:41.919832 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.919817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 17:02:41.926538 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.926511    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"]
Aug 03 17:02:41.952964 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:41.952937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419947de-7c72-464c-bed5-0d9bcd8423b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8\" (UID: \"419947de-7c72-464c-bed5-0d9bcd8423b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"
Aug 03 17:02:42.054143 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:42.054109    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419947de-7c72-464c-bed5-0d9bcd8423b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8\" (UID: \"419947de-7c72-464c-bed5-0d9bcd8423b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"
Aug 03 17:02:42.054766 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:42.054723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419947de-7c72-464c-bed5-0d9bcd8423b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8\" (UID: \"419947de-7c72-464c-bed5-0d9bcd8423b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"
Aug 03 17:02:42.226923 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:42.226844    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"
Aug 03 17:02:42.356814 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:42.356788    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"]
Aug 03 17:02:42.358515 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:02:42.358483    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419947de_7c72_464c_bed5_0d9bcd8423b3.slice/crio-41c9e43bf7d83c07fc60a3e981a9fad60a2baceaa588aa265ce418f5e29258c3 WatchSource:0}: Error finding container 41c9e43bf7d83c07fc60a3e981a9fad60a2baceaa588aa265ce418f5e29258c3: Status 404 returned error can't find the container with id 41c9e43bf7d83c07fc60a3e981a9fad60a2baceaa588aa265ce418f5e29258c3
Aug 03 17:02:43.097653 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:43.097608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8" event={"ID":"419947de-7c72-464c-bed5-0d9bcd8423b3","Type":"ContainerStarted","Data":"193a53fff3d211840399c9f2b5b7b82621ba02442d1db8121066c4c987f608d7"}
Aug 03 17:02:43.098075 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:43.097673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8" event={"ID":"419947de-7c72-464c-bed5-0d9bcd8423b3","Type":"ContainerStarted","Data":"41c9e43bf7d83c07fc60a3e981a9fad60a2baceaa588aa265ce418f5e29258c3"}
Aug 03 17:02:43.114914 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:43.114860    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8" podStartSLOduration=2.114841999 podStartE2EDuration="2.114841999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:02:43.113296734 +0000 UTC m=+3505.380112745" watchObservedRunningTime="2026-08-03 17:02:43.114841999 +0000 UTC m=+3505.381658007"
Aug 03 17:02:44.108562 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:44.108529    2564 generic.go:356] "Generic (PLEG): container finished" podID="419947de-7c72-464c-bed5-0d9bcd8423b3" containerID="193a53fff3d211840399c9f2b5b7b82621ba02442d1db8121066c4c987f608d7" exitCode=0
Aug 03 17:02:44.108964 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:44.108619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8" event={"ID":"419947de-7c72-464c-bed5-0d9bcd8423b3","Type":"ContainerDied","Data":"193a53fff3d211840399c9f2b5b7b82621ba02442d1db8121066c4c987f608d7"}
Aug 03 17:02:45.237604 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:45.237582    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"
Aug 03 17:02:45.284247 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:45.284220    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419947de-7c72-464c-bed5-0d9bcd8423b3-maas-api-service-ca\") pod \"419947de-7c72-464c-bed5-0d9bcd8423b3\" (UID: \"419947de-7c72-464c-bed5-0d9bcd8423b3\") "
Aug 03 17:02:45.284572 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:45.284543    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/419947de-7c72-464c-bed5-0d9bcd8423b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "419947de-7c72-464c-bed5-0d9bcd8423b3" (UID: "419947de-7c72-464c-bed5-0d9bcd8423b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:02:45.385242 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:45.385167    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419947de-7c72-464c-bed5-0d9bcd8423b3-maas-api-service-ca\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:02:46.119294 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:46.119268    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"
Aug 03 17:02:46.119498 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:46.119306    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8" event={"ID":"419947de-7c72-464c-bed5-0d9bcd8423b3","Type":"ContainerDied","Data":"41c9e43bf7d83c07fc60a3e981a9fad60a2baceaa588aa265ce418f5e29258c3"}
Aug 03 17:02:46.119498 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:46.119364    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41c9e43bf7d83c07fc60a3e981a9fad60a2baceaa588aa265ce418f5e29258c3"
Aug 03 17:02:46.150183 ip-10-0-138-110 kubenswrapper[2564]: E0803 17:02:46.150152    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419947de_7c72_464c_bed5_0d9bcd8423b3.slice\": RecentStats: unable to find data in memory cache]"
Aug 03 17:02:52.337530 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.337496    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"]
Aug 03 17:02:52.344104 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.344076    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-cf8ed9-29762940-tg8wt"]
Aug 03 17:02:52.366453 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.366425    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"]
Aug 03 17:02:52.366639 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.366614    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5" podUID="f97c998f-e118-4fc7-8817-b78aa0fd28d2" containerName="payload-processing" containerID="cri-o://6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2" gracePeriod=30
Aug 03 17:02:52.538554 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.538528    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"]
Aug 03 17:02:52.541123 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.541096    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cf8ed9-25mv8"]
Aug 03 17:02:52.614275 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.614252    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 17:02:52.650685 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.650658    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f97c998f-e118-4fc7-8817-b78aa0fd28d2-plugins-config-volume\") pod \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") "
Aug 03 17:02:52.650871 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.650701    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbt5c\" (UniqueName: \"kubernetes.io/projected/f97c998f-e118-4fc7-8817-b78aa0fd28d2-kube-api-access-pbt5c\") pod \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\" (UID: \"f97c998f-e118-4fc7-8817-b78aa0fd28d2\") "
Aug 03 17:02:52.651110 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.651084    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f97c998f-e118-4fc7-8817-b78aa0fd28d2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f97c998f-e118-4fc7-8817-b78aa0fd28d2" (UID: "f97c998f-e118-4fc7-8817-b78aa0fd28d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:02:52.652727 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.652705    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f97c998f-e118-4fc7-8817-b78aa0fd28d2-kube-api-access-pbt5c" (OuterVolumeSpecName: "kube-api-access-pbt5c") pod "f97c998f-e118-4fc7-8817-b78aa0fd28d2" (UID: "f97c998f-e118-4fc7-8817-b78aa0fd28d2"). InnerVolumeSpecName "kube-api-access-pbt5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:02:52.752318 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.752282    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f97c998f-e118-4fc7-8817-b78aa0fd28d2-plugins-config-volume\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:02:52.752318 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:52.752312    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbt5c\" (UniqueName: \"kubernetes.io/projected/f97c998f-e118-4fc7-8817-b78aa0fd28d2-kube-api-access-pbt5c\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:02:53.152861 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.152823    2564 generic.go:356] "Generic (PLEG): container finished" podID="f97c998f-e118-4fc7-8817-b78aa0fd28d2" containerID="6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2" exitCode=0
Aug 03 17:02:53.153045 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.152869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5" event={"ID":"f97c998f-e118-4fc7-8817-b78aa0fd28d2","Type":"ContainerDied","Data":"6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2"}
Aug 03 17:02:53.153045 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.152911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5" event={"ID":"f97c998f-e118-4fc7-8817-b78aa0fd28d2","Type":"ContainerDied","Data":"102de57a41765ca4b41ad848281f35bfadf1ade662172e3c51a0cfe31809da2e"}
Aug 03 17:02:53.153045 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.152934    2564 scope.go:117] "RemoveContainer" containerID="6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2"
Aug 03 17:02:53.153045 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.152873    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"
Aug 03 17:02:53.162352 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.162330    2564 scope.go:117] "RemoveContainer" containerID="6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2"
Aug 03 17:02:53.162618 ip-10-0-138-110 kubenswrapper[2564]: E0803 17:02:53.162598    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2\": container with ID starting with 6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2 not found: ID does not exist" containerID="6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2"
Aug 03 17:02:53.162691 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.162629    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2"} err="failed to get container status \"6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2\": rpc error: code = NotFound desc = could not find container \"6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2\": container with ID starting with 6a0c2d6b9059a16d464f11b74619713e76d3f2ae782b98a0d9a24b53f9ae86b2 not found: ID does not exist"
Aug 03 17:02:53.175109 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.175087    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"]
Aug 03 17:02:53.179422 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:53.179403    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cf8ed9-7d67bdb79d-45gv5"]
Aug 03 17:02:54.242687 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:54.242646    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="419947de-7c72-464c-bed5-0d9bcd8423b3" path="/var/lib/kubelet/pods/419947de-7c72-464c-bed5-0d9bcd8423b3/volumes"
Aug 03 17:02:54.243182 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:54.242990    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="868f0af5-ae92-490b-ab80-38ae69445c92" path="/var/lib/kubelet/pods/868f0af5-ae92-490b-ab80-38ae69445c92/volumes"
Aug 03 17:02:54.243300 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:54.243285    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f97c998f-e118-4fc7-8817-b78aa0fd28d2" path="/var/lib/kubelet/pods/f97c998f-e118-4fc7-8817-b78aa0fd28d2/volumes"
Aug 03 17:02:54.592708 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:02:54.592673    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:03:00.058138 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.058103    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"]
Aug 03 17:03:00.058516 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.058418    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f97c998f-e118-4fc7-8817-b78aa0fd28d2" containerName="payload-processing"
Aug 03 17:03:00.058516 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.058428    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f97c998f-e118-4fc7-8817-b78aa0fd28d2" containerName="payload-processing"
Aug 03 17:03:00.058516 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.058443    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="419947de-7c72-464c-bed5-0d9bcd8423b3" containerName="revoke-keys"
Aug 03 17:03:00.058516 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.058449    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="419947de-7c72-464c-bed5-0d9bcd8423b3" containerName="revoke-keys"
Aug 03 17:03:00.058516 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.058498    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="419947de-7c72-464c-bed5-0d9bcd8423b3" containerName="revoke-keys"
Aug 03 17:03:00.058516 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.058508    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f97c998f-e118-4fc7-8817-b78aa0fd28d2" containerName="payload-processing"
Aug 03 17:03:00.062776 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.062758    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"
Aug 03 17:03:00.065632 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.065603    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 17:03:00.065797 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.065705    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 03 17:03:00.069110 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.069086    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"]
Aug 03 17:03:00.118490 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.118462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a04e348-780b-4aa7-a3ae-14bcb8023c10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj\" (UID: \"9a04e348-780b-4aa7-a3ae-14bcb8023c10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"
Aug 03 17:03:00.219311 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.219275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a04e348-780b-4aa7-a3ae-14bcb8023c10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj\" (UID: \"9a04e348-780b-4aa7-a3ae-14bcb8023c10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"
Aug 03 17:03:00.219981 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.219964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a04e348-780b-4aa7-a3ae-14bcb8023c10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj\" (UID: \"9a04e348-780b-4aa7-a3ae-14bcb8023c10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"
Aug 03 17:03:00.374290 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.374211    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"
Aug 03 17:03:00.502618 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:00.502591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"]
Aug 03 17:03:00.504488 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:03:00.504461    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a04e348_780b_4aa7_a3ae_14bcb8023c10.slice/crio-09691d78b67cb9d3e80d639a4fde2ece7a0819c366e7fce5b1362a2ea1426880 WatchSource:0}: Error finding container 09691d78b67cb9d3e80d639a4fde2ece7a0819c366e7fce5b1362a2ea1426880: Status 404 returned error can't find the container with id 09691d78b67cb9d3e80d639a4fde2ece7a0819c366e7fce5b1362a2ea1426880
Aug 03 17:03:01.198125 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:01.198092    2564 generic.go:356] "Generic (PLEG): container finished" podID="9a04e348-780b-4aa7-a3ae-14bcb8023c10" containerID="f7db20fef291b49e9f8a5af649fc3313a3691b8bb0dc6befc24ecb87532a7f53" exitCode=0
Aug 03 17:03:01.198513 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:01.198167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj" event={"ID":"9a04e348-780b-4aa7-a3ae-14bcb8023c10","Type":"ContainerDied","Data":"f7db20fef291b49e9f8a5af649fc3313a3691b8bb0dc6befc24ecb87532a7f53"}
Aug 03 17:03:01.198513 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:01.198203    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj" event={"ID":"9a04e348-780b-4aa7-a3ae-14bcb8023c10","Type":"ContainerStarted","Data":"09691d78b67cb9d3e80d639a4fde2ece7a0819c366e7fce5b1362a2ea1426880"}
Aug 03 17:03:02.325874 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:02.325847    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"
Aug 03 17:03:02.438648 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:02.438613    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a04e348-780b-4aa7-a3ae-14bcb8023c10-maas-api-service-ca\") pod \"9a04e348-780b-4aa7-a3ae-14bcb8023c10\" (UID: \"9a04e348-780b-4aa7-a3ae-14bcb8023c10\") "
Aug 03 17:03:02.439024 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:02.439004    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a04e348-780b-4aa7-a3ae-14bcb8023c10-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a04e348-780b-4aa7-a3ae-14bcb8023c10" (UID: "9a04e348-780b-4aa7-a3ae-14bcb8023c10"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:02.539751 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:02.539660    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a04e348-780b-4aa7-a3ae-14bcb8023c10-maas-api-service-ca\") on node \"ip-10-0-138-110.ec2.internal\" DevicePath \"\""
Aug 03 17:03:03.209271 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:03.209236    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"
Aug 03 17:03:03.209271 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:03.209248    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj" event={"ID":"9a04e348-780b-4aa7-a3ae-14bcb8023c10","Type":"ContainerDied","Data":"09691d78b67cb9d3e80d639a4fde2ece7a0819c366e7fce5b1362a2ea1426880"}
Aug 03 17:03:03.209498 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:03.209330    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09691d78b67cb9d3e80d639a4fde2ece7a0819c366e7fce5b1362a2ea1426880"
Aug 03 17:03:10.639620 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:10.639584    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"]
Aug 03 17:03:10.644566 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:10.644540    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0fae8e-29762940-mp4qz"]
Aug 03 17:03:10.803477 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:10.803442    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"]
Aug 03 17:03:10.807309 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:10.807287    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0fae8e-6pbcj"]
Aug 03 17:03:12.242724 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:12.242691    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a04e348-780b-4aa7-a3ae-14bcb8023c10" path="/var/lib/kubelet/pods/9a04e348-780b-4aa7-a3ae-14bcb8023c10/volumes"
Aug 03 17:03:12.243120 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:12.242995    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3d28d34-45c2-4d2c-8f1e-27c7bd226896" path="/var/lib/kubelet/pods/b3d28d34-45c2-4d2c-8f1e-27c7bd226896/volumes"
Aug 03 17:03:12.791075 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:12.791040    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:03:29.026969 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:29.026934    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"]
Aug 03 17:03:29.031838 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:29.031812    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7055f0-29762940-d56lx"]
Aug 03 17:03:30.243471 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:30.243442    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1afe3b3-216e-4c93-93e5-5d382c7fe832" path="/var/lib/kubelet/pods/a1afe3b3-216e-4c93-93e5-5d382c7fe832/volumes"
Aug 03 17:03:30.993267 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:30.993233    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-kjlq6"]
Aug 03 17:03:44.460280 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:44.460248    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qz2d8_c7324a4a-a439-4830-966f-41bcb5d1023d/manager/0.log"
Aug 03 17:03:44.923041 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:44.923015    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-jd9rw_922e9513-26ab-4ed4-8892-86b775b00ad0/copy-manifests/0.log"
Aug 03 17:03:44.935963 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:44.935939    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-jd9rw_922e9513-26ab-4ed4-8892-86b775b00ad0/manager/0.log"
Aug 03 17:03:45.694234 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:45.694206    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29762925-v8qwp_b963e35b-db24-498f-8aff-d11668886f25/cleanup/0.log"
Aug 03 17:03:45.832868 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:45.832834    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29762940-zqbc4_589e4a24-c479-49f4-a137-ff7e8eb6e302/cleanup/0.log"
Aug 03 17:03:47.340158 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:47.340135    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-l2vkb_32938926-5f06-423d-918f-bded0ddf7d64/kuadrant-console-plugin/0.log"
Aug 03 17:03:47.631852 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:47.631754    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-kjlq6_c33cda60-e479-4f41-b5f8-9f7887908846/limitador/0.log"
Aug 03 17:03:47.770418 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:47.770385    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-278cz_0e66f32c-ba10-4836-871f-a132ee22a26c/manager/0.log"
Aug 03 17:03:48.508956 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:48.508920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-q96jk_607ce5d7-90c3-4174-ad8d-8ab698fea2fe/kube-auth-proxy/0.log"
Aug 03 17:03:50.264921 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:50.264891    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-zhmbp_37a421d1-78f8-478a-9764-8070dd403b14/main/0.log"
Aug 03 17:03:50.414672 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:03:50.414640    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lwhft_b3956d23-c892-4558-bc02-36deb60e0039/main/0.log"
Aug 03 17:04:00.526352 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:00.526319    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5nw44_c18f9f7f-3374-4214-b9e8-a98a23f5cc98/global-pull-secret-syncer/0.log"
Aug 03 17:04:00.705643 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:00.705615    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jkhtn_79aebf44-1875-4f4e-a708-c21df0257b1d/konnectivity-agent/0.log"
Aug 03 17:04:00.793284 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:00.793205    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-110.ec2.internal_ab1f6e49c95683c2cc3b4fd509ec7a98/haproxy/0.log"
Aug 03 17:04:05.541331 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:05.541302    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-l2vkb_32938926-5f06-423d-918f-bded0ddf7d64/kuadrant-console-plugin/0.log"
Aug 03 17:04:05.713720 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:05.713686    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-kjlq6_c33cda60-e479-4f41-b5f8-9f7887908846/limitador/0.log"
Aug 03 17:04:05.796953 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:05.796860    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-278cz_0e66f32c-ba10-4836-871f-a132ee22a26c/manager/0.log"
Aug 03 17:04:07.659098 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:07.659070    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-785f6df764-q8vqj_f4e83f09-5e58-4e5a-8416-262e1f876f2f/metrics-server/0.log"
Aug 03 17:04:07.698754 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:07.698719    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-zkvmx_4381cea8-6c83-482c-8809-34e69fc0f355/monitoring-plugin/0.log"
Aug 03 17:04:07.739648 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:07.739615    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-98mh7_923f8f36-9a36-4b9d-a8c9-32c7787e7443/node-exporter/0.log"
Aug 03 17:04:07.773059 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:07.773033    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-98mh7_923f8f36-9a36-4b9d-a8c9-32c7787e7443/kube-rbac-proxy/0.log"
Aug 03 17:04:07.809990 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:07.809962    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-98mh7_923f8f36-9a36-4b9d-a8c9-32c7787e7443/init-textfile/0.log"
Aug 03 17:04:08.149781 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.149753    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a56f11f-47f6-440f-9071-f0fd0d910489/prometheus/0.log"
Aug 03 17:04:08.177123 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.177097    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a56f11f-47f6-440f-9071-f0fd0d910489/config-reloader/0.log"
Aug 03 17:04:08.202961 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.202936    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a56f11f-47f6-440f-9071-f0fd0d910489/thanos-sidecar/0.log"
Aug 03 17:04:08.230876 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.230852    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a56f11f-47f6-440f-9071-f0fd0d910489/kube-rbac-proxy-web/0.log"
Aug 03 17:04:08.260426 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.260402    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a56f11f-47f6-440f-9071-f0fd0d910489/kube-rbac-proxy/0.log"
Aug 03 17:04:08.295000 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.294971    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a56f11f-47f6-440f-9071-f0fd0d910489/kube-rbac-proxy-thanos/0.log"
Aug 03 17:04:08.324123 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.324095    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0a56f11f-47f6-440f-9071-f0fd0d910489/init-config-reloader/0.log"
Aug 03 17:04:08.362532 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.362500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-65jb7_daca8c9e-b40e-4c48-98ce-d3c3dc6d692e/prometheus-operator/0.log"
Aug 03 17:04:08.391061 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:08.391033    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-65jb7_daca8c9e-b40e-4c48-98ce-d3c3dc6d692e/kube-rbac-proxy/0.log"
Aug 03 17:04:10.297241 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.297158    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"]
Aug 03 17:04:10.297610 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.297485    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a04e348-780b-4aa7-a3ae-14bcb8023c10" containerName="revoke-keys"
Aug 03 17:04:10.297610 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.297496    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a04e348-780b-4aa7-a3ae-14bcb8023c10" containerName="revoke-keys"
Aug 03 17:04:10.297610 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.297561    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a04e348-780b-4aa7-a3ae-14bcb8023c10" containerName="revoke-keys"
Aug 03 17:04:10.300382 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.300365    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.303290 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.303270    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpjhq\"/\"kube-root-ca.crt\""
Aug 03 17:04:10.304414 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.304390    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpjhq\"/\"openshift-service-ca.crt\""
Aug 03 17:04:10.304535 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.304396    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xpjhq\"/\"default-dockercfg-dl5pw\""
Aug 03 17:04:10.308454 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.308428    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"]
Aug 03 17:04:10.423977 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.423939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw4zq\" (UniqueName: \"kubernetes.io/projected/28acc25e-6ef7-4ecf-aebf-ad59309cf668-kube-api-access-gw4zq\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.423977 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.423982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-lib-modules\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.424203 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.424017    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-sys\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.424203 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.424066    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-proc\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.424203 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.424113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-podres\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525313 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525277    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-proc\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525313 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-podres\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525550 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw4zq\" (UniqueName: \"kubernetes.io/projected/28acc25e-6ef7-4ecf-aebf-ad59309cf668-kube-api-access-gw4zq\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525550 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-lib-modules\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525550 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525420    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-proc\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525550 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-sys\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525550 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-sys\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525550 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525491    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-podres\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.525550 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.525522    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28acc25e-6ef7-4ecf-aebf-ad59309cf668-lib-modules\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.535189 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.535153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw4zq\" (UniqueName: \"kubernetes.io/projected/28acc25e-6ef7-4ecf-aebf-ad59309cf668-kube-api-access-gw4zq\") pod \"perf-node-gather-daemonset-fcd5m\" (UID: \"28acc25e-6ef7-4ecf-aebf-ad59309cf668\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.612166 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.612131    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:10.749347 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:10.749317    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"]
Aug 03 17:04:10.751791 ip-10-0-138-110 kubenswrapper[2564]: W0803 17:04:10.751759    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod28acc25e_6ef7_4ecf_aebf_ad59309cf668.slice/crio-7b62384928e3a1604a9ca7ee12cbd9bbc96c2185c840d8b9daac1db5f11ba54a WatchSource:0}: Error finding container 7b62384928e3a1604a9ca7ee12cbd9bbc96c2185c840d8b9daac1db5f11ba54a: Status 404 returned error can't find the container with id 7b62384928e3a1604a9ca7ee12cbd9bbc96c2185c840d8b9daac1db5f11ba54a
Aug 03 17:04:11.155548 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:11.155520    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-fbcc56d8f-9w6v5_24cdd403-0b54-40d6-8522-65f12b3088fb/console/0.log"
Aug 03 17:04:11.521067 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:11.520981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m" event={"ID":"28acc25e-6ef7-4ecf-aebf-ad59309cf668","Type":"ContainerStarted","Data":"07331864e7b60be8c3c4e1096db4c8819704a6bae0bc52de03b464ea8c1fb636"}
Aug 03 17:04:11.521067 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:11.521031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m" event={"ID":"28acc25e-6ef7-4ecf-aebf-ad59309cf668","Type":"ContainerStarted","Data":"7b62384928e3a1604a9ca7ee12cbd9bbc96c2185c840d8b9daac1db5f11ba54a"}
Aug 03 17:04:11.521505 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:11.521114    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:11.540419 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:11.540368    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m" podStartSLOduration=1.5403511829999998 podStartE2EDuration="1.540351183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 17:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:04:11.539686328 +0000 UTC m=+3593.806502348" watchObservedRunningTime="2026-08-03 17:04:11.540351183 +0000 UTC m=+3593.807167189"
Aug 03 17:04:12.968546 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:12.968508    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zd6c_6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc/dns/0.log"
Aug 03 17:04:13.018449 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:13.018419    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zd6c_6fbcfd14-0820-4d90-a3a8-9a9d62b5f1fc/kube-rbac-proxy/0.log"
Aug 03 17:04:13.248702 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:13.248620    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2dtgs_e16925c5-9e44-4b01-bb5d-fd1172662359/dns-node-resolver/0.log"
Aug 03 17:04:13.910585 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:13.910557    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-58568488cc-mxl9t_f52dae94-350b-4c29-86a5-c8cd5440f5c7/registry/0.log"
Aug 03 17:04:13.991298 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:13.991269    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-69qfd_c244ee48-8f52-4a87-92cd-df1c00b2d8ca/node-ca/0.log"
Aug 03 17:04:15.437549 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:15.437502    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-q96jk_607ce5d7-90c3-4174-ad8d-8ab698fea2fe/kube-auth-proxy/0.log"
Aug 03 17:04:16.523023 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:16.522992    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mp7kr_13857d42-267a-40b4-959a-85e751549e0b/serve-healthcheck-canary/0.log"
Aug 03 17:04:17.263533 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:17.263500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wg65w_45cbc0e2-f246-44a8-a38b-ed39d24ac032/kube-rbac-proxy/0.log"
Aug 03 17:04:17.286888 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:17.286852    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wg65w_45cbc0e2-f246-44a8-a38b-ed39d24ac032/exporter/0.log"
Aug 03 17:04:17.310353 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:17.310315    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wg65w_45cbc0e2-f246-44a8-a38b-ed39d24ac032/extractor/0.log"
Aug 03 17:04:17.535970 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:17.535892    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-fcd5m"
Aug 03 17:04:18.385981 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:18.385954    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 17:04:18.395078 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:18.395056    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mprtd_1fe44827-6e6a-4c1e-a105-a6734ddbb548/ovn-acl-logging/0.log"
Aug 03 17:04:20.221774 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:20.221731    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qz2d8_c7324a4a-a439-4830-966f-41bcb5d1023d/manager/0.log"
Aug 03 17:04:20.434573 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:20.434528    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-jd9rw_922e9513-26ab-4ed4-8892-86b775b00ad0/manager/0.log"
Aug 03 17:04:20.456519 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:20.456495    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-jd9rw_922e9513-26ab-4ed4-8892-86b775b00ad0/copy-manifests/0.log"
Aug 03 17:04:20.862426 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:20.862397    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-q96jk_607ce5d7-90c3-4174-ad8d-8ab698fea2fe/kube-auth-proxy/0.log"
Aug 03 17:04:21.724592 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:21.724513    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7596f8d454-8r7jv_b8c8103b-b410-4f60-9371-66a280a69600/manager/0.log"
Aug 03 17:04:22.661306 ip-10-0-138-110 kubenswrapper[2564]: I0803 17:04:22.661279    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-q96jk_607ce5d7-90c3-4174-ad8d-8ab698fea2fe/kube-auth-proxy/0.log"