Jul 17 17:54:38.809876 ip-10-0-140-23 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 17:54:38.809893 ip-10-0-140-23 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 17:54:38.809903 ip-10-0-140-23 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 17:54:38.810371 ip-10-0-140-23 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 17:54:49.059239 ip-10-0-140-23 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 17:54:49.059260 ip-10-0-140-23 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 609159b06de2454b84cbc9c65766c6e2 --
Jul 17 17:56:42.246029 ip-10-0-140-23 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 17:56:42.699284 ip-10-0-140-23 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 17:56:42.699284 ip-10-0-140-23 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 17:56:42.699284 ip-10-0-140-23 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 17:56:42.699284 ip-10-0-140-23 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 17:56:42.699284 ip-10-0-140-23 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 17:56:42.700214 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.700085    2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 17:56:42.705430 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705403    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:56:42.705430 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705425    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:56:42.705430 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705431    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:56:42.705430 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705434    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:56:42.705430 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705438    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705441    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705444    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705446    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705449    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705454    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705456    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705459    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705462    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705464    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705468    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705470    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705473    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705476    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705479    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705482    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705485    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705487    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705490    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705493    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:56:42.705638 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705495    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705498    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705501    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705508    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705511    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705514    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705516    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705519    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705522    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705526    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705530    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705533    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705535    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705539    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705543    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705546    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705549    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705552    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705555    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705558    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:56:42.706181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705561    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705564    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705567    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705571    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705574    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705577    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705580    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705583    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705586    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705589    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705591    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705594    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705597    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705600    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705603    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705605    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705608    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705611    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705614    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705616    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:56:42.706707 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705619    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705622    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705625    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705627    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705630    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705633    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705637    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705641    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705647    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705652    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705656    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705659    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705662    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705665    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705667    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705670    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705673    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705676    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705678    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:56:42.707235 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705682    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705685    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705687    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705690    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705693    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705695    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705699    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705702    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705704    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705707    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705710    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705728    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705731    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705734    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705737    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.705740    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706508    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706515    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706518    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706522    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:56:42.707731 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706525    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706528    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706531    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706534    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706537    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706540    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706543    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706546    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706548    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706552    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706606    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706611    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706615    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706619    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706623    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706626    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706629    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706632    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706635    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706638    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:56:42.708273 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706641    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706644    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706647    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706649    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706652    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706655    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706658    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706660    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706663    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706666    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706669    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706672    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706675    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706678    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706681    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706684    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706687    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706690    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706692    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706695    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:56:42.708843 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706699    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706701    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706704    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706707    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706709    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706729    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706733    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706738    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706742    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706745    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706748    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706751    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706765    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706768    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706771    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706774    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706777    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706780    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706782    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:56:42.709371 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706787    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706790    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706794    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706797    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706801    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706804    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706807    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706810    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706813    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706816    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706818    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706821    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706824    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706826    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706829    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706832    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706835    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706838    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706841    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:56:42.709894 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706844    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706847    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706849    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706852    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706855    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706857    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706860    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706864    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706868    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706870    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706874    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706876    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706879    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706882    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706884    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706887    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.706890    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707331    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707338    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:56:42.710379 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707342    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707346    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707349    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707353    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707356    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707359    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707361    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707364    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707367    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707370    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707373    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707375    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707379    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707381    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707385    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707387    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707390    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707394    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707396    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707399    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:56:42.710895 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707402    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707404    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707409    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707411    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707414    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707416    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707419    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707422    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707425    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707428    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707431    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707433    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707436    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707439    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707441    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707444    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707447    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707450    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707452    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707455    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:56:42.711418 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707458    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707461    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707465    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707468    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707471    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707474    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707476    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707480    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707482    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707485    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707488    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707490    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707493    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707496    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707500    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707503    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707506    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707509    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707512    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707515    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:56:42.711947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707517    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707521    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707523    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707526    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707529    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707532    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707535    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707537    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707540    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707543    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707546    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707548    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707551    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707554    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707556    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707559    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707562    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707565    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707567    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707571    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:56:42.712461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707574    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707577    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707579    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707582    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707585    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707587    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707590    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707596    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707599    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707602    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707605    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707607    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707610    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707613    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707616    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707618    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.707621    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707706    2555 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707728    2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707736    2555 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 17:56:42.712980 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707741    2555 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707746    2555 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707749    2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707754    2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707759    2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707762    2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707765    2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707769    2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707772    2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707775    2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707779    2555 flags.go:64] FLAG: --cgroup-root=""
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707782    2555 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707786    2555 flags.go:64] FLAG: --client-ca-file=""
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707789    2555 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707793    2555 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707797    2555 flags.go:64] FLAG: --cluster-domain=""
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707800    2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707803    2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707806    2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707813    2555 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707817    2555 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707820    2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707823    2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 17:56:42.713491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707826    2555 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707830    2555 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707833    2555 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707836    2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707839    2555 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707843    2555 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707850    2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707855    2555 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707859    2555 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707864    2555 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707869    2555 flags.go:64] FLAG: --enable-server="true"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707874    2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707879    2555 flags.go:64] FLAG: --event-burst="100"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707883    2555 flags.go:64] FLAG: --event-qps="50"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707886    2555 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707889    2555 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707892    2555 flags.go:64] FLAG: --eviction-hard=""
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707896    2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707899    2555 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707902    2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707906    2555 flags.go:64] FLAG: --eviction-soft=""
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707909    2555 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707911    2555 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707915    2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707918    2555 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 17:56:42.714100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707922    2555 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707925    2555 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707928    2555 flags.go:64] FLAG: --feature-gates=""
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707935    2555 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707939    2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707942    2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707946    2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707949    2555 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707952    2555 flags.go:64] FLAG: --help="false"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707955    2555 flags.go:64] FLAG: --hostname-override="ip-10-0-140-23.ec2.internal"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707958    2555 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707961    2555 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707964    2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707968    2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707971    2555 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707974    2555 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707978    2555 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707980    2555 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707983    2555 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707986    2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707990    2555 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707993    2555 flags.go:64] FLAG: --kube-reserved=""
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707996    2555 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.707998    2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 17:56:42.714817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708002    2555 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708005    2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708008    2555 flags.go:64] FLAG: --lock-file=""
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708011    2555 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708014    2555 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708017    2555 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708023    2555 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708026    2555 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708030    2555 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708032    2555 flags.go:64] FLAG: --logging-format="text"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708036    2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708041    2555 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708044    2555 flags.go:64] FLAG: --manifest-url=""
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708047    2555 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708052    2555 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708055    2555 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708059    2555 flags.go:64] FLAG: --max-pods="110"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708062    2555 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708065    2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708068    2555 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708072    2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708075    2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708078    2555 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708081    2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708089    2555 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 17:56:42.715775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708092    2555 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708095    2555 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708098    2555 flags.go:64] FLAG: --pod-cidr=""
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708101    2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708107    2555 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708110    2555 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708113    2555 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708116    2555 flags.go:64] FLAG: --port="10250"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708119    2555 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708122    2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f9e92367c90b0095"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708126    2555 flags.go:64] FLAG: --qos-reserved=""
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708129    2555 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708132    2555 flags.go:64] FLAG: --register-node="true"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708135    2555 flags.go:64] FLAG: --register-with-taints=""
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708138    2555 flags.go:64] FLAG: --registry-burst="10"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708142    2555 flags.go:64] FLAG: --registry-qps="5"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708145    2555 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708148    2555 flags.go:64] FLAG: --reserved-memory=""
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708152    2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708156    2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708160    2555 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708163    2555 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708166    2555 flags.go:64] FLAG: --runonce="false"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708169    2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708172    2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 17:56:42.716921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708175    2555 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708178    2555 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708181    2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708184    2555 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708187    2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708191    2555 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708194    2555 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708197    2555 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708200    2555 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708203    2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708206    2555 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708209    2555 flags.go:64] FLAG: --system-cgroups=""
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708212    2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708218    2555 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708221    2555 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708224    2555 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708229    2555 flags.go:64] FLAG: --tls-min-version=""
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708232    2555 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708234    2555 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708238    2555 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708241    2555 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708244    2555 flags.go:64] FLAG: --v="2"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708254    2555 flags.go:64] FLAG: --version="false"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708259    2555 flags.go:64] FLAG: --vmodule=""
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708265    2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 17:56:42.718216 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.708268    2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708395    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708399    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708402    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708405    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708408    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708411    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708414    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708417    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708419    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708422    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708425    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708428    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708430    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708433    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708436    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708439    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708442    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708444    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708447    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:56:42.719360 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708450    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708453    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708456    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708459    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708461    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708464    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708467    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708470    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708473    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708475    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708478    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708483    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708486    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708490    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708492    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708495    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708498    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708500    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708503    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708506    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:56:42.720008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708509    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708512    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708515    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708517    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708520    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708522    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708525    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708530    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708533    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708536    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708539    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708542    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708545    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708548    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708550    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708553    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708556    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708559    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708561    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:56:42.720972 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708564    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708567    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708569    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708572    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708576    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708579    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708584    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708588    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708590    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708593    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708596    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708599    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708602    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708605    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708607    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708610    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708614    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708618    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708621    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708624    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:56:42.722007 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708626    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708629    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708632    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708635    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708638    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708640    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708644    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708648    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708651    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708654    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708657    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708660    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708663    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708666    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708669    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708672    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708677    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708680    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708686    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:56:42.722697 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708689    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.708692    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.709507    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.717601    2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.717626    2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.717656    2555 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.717667    2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717800    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717811    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717815    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717820    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717826    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717832    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717837    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:56:42.723255 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717842    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717846    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717851    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717856    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717860    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717865    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717870    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717875    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717880    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717884    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717888    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717893    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717901    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717907    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717912    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717919    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717924    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717929    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717935    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:56:42.723632 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717940    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717945    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717950    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717958    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717963    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717968    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717973    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717977    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717981    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717987    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717992    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.717996    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718001    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718005    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718010    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718016    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718020    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718025    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718029    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718035    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:56:42.724203 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718039    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718045    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718049    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718054    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718058    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718063    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718067    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718072    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718077    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718081    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718086    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718091    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718096    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718101    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718105    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718111    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718116    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718121    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718125    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718130    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:56:42.724835 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718135    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718139    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718144    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718148    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718153    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718157    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718162    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718166    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718171    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718176    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718181    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718185    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718190    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718194    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718199    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718204    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718208    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718212    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718217    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718222    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:56:42.725393 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718226    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718231    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718235    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718240    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718244    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718249    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718253    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718259    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718263    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718269    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718274    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718279    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718284    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.718294    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718523    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 17:56:42.726119 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718532    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718537    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718541    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718546    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718550    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718555    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718559    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718564    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718569    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718573    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718578    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718583    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718587    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718591    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718596    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718600    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718605    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718611    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718616    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718621    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 17:56:42.726517 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718625    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718630    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718635    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718639    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718656    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718661    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718666    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718681    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718686    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718690    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718695    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718700    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718704    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718709    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718731    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718736    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718741    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718746    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718750    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718754    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 17:56:42.727275 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718758    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718763    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718767    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718775    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718782    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718787    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718793    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718798    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718803    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718807    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718813    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718818    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718823    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718828    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718833    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718840    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718846    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718850    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718855    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 17:56:42.728156 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718859    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718873    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718879    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718883    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718887    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718892    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718897    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718901    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718905    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718910    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718914    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718919    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718923    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718927    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718932    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718936    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718941    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718945    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718950    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718954    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 17:56:42.729006 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718960    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718965    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718969    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718976    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718982    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718987    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718992    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.718997    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719002    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719007    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719012    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719017    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719023    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719027    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719040    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719045    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719050    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719054    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 17:56:42.729745 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:42.719059    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 17:56:42.730465 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.719068    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 17:56:42.730465 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.719279    2555 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 17:56:42.730465 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.722246    2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 17:56:42.730465 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.723767    2555 server.go:1019] "Starting client certificate rotation"
Jul 17 17:56:42.730465 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.723893    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 17:56:42.730465 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.723934    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 17:56:42.730465 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.727284    2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 17:56:42.732203 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.732179    2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 17:56:42.744832 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.744801    2555 log.go:25] "Validated CRI v1 runtime API"
Jul 17 17:56:42.746878 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.746859    2555 log.go:25] "Validated CRI v1 image API"
Jul 17 17:56:42.748471 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.748450    2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 17:56:42.752705 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.752684    2555 fs.go:135] Filesystem UUIDs: map[5fe83db6-6f61-461c-8e3c-1111e25d867d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cb17eeb8-80aa-454b-b1b6-9ba583b6fb5f:/dev/nvme0n1p4]
Jul 17 17:56:42.752774 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.752704    2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 17:56:42.756943 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.756920    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 17:56:42.759097 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.758974    2555 manager.go:216] Machine: {Timestamp:2026-07-17 17:56:42.757560824 +0000 UTC m=+0.343037070 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3127026 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29dfec93bb87fb3292336dc17f4b52 SystemUUID:ec29dfec-93bb-87fb-3292-336dc17f4b52 BootID:609159b0-6de2-454b-84cb-c9c65766c6e2 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e3:ab:61:c2:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e3:ab:61:c2:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:5f:97:69:f5:98 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 17:56:42.759097 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.759098    2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 17:56:42.759213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.759198    2555 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 17:56:42.761951 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.761919    2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 17:56:42.762109 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.761954    2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-23.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 17:56:42.762754 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.762744    2555 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 17:56:42.762797 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.762757    2555 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 17:56:42.762797 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.762765    2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 17:56:42.763774 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.763762    2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 17:56:42.764935 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.764924    2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 17:56:42.767512 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.767499    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 17:56:42.767659 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.767649    2555 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 17:56:42.767691 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.767684    2555 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 17:56:42.767743 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.767699    2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 17:56:42.767743 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.767711    2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 17:56:42.767743 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.767738    2555 kubelet.go:398] "Adding apiserver pod source"
Jul 17 17:56:42.767824 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.767763    2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 17:56:42.768265 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.768233    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 17:56:42.768347 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.768272    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 17:56:42.770781 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.770758    2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 17:56:42.772657 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.772639    2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 17:56:42.772711 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.772681    2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 17:56:42.773866 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773853    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 17:56:42.773902 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773871    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 17:56:42.773902 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773878    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 17:56:42.773902 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773883    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 17:56:42.773902 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773892    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 17:56:42.773902 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773898    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 17:56:42.774041 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773906    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 17:56:42.774041 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773912    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 17:56:42.774041 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773919    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 17:56:42.774041 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773924    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 17:56:42.774041 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773934    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 17:56:42.774041 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.773941    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 17:56:42.774797 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.774783    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 17:56:42.774836 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.774800    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 17:56:42.779424 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.779403    2555 server.go:1268] "Started kubelet"
Jul 17 17:56:42.779493 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.779425    2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-23.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 17:56:42.780133 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.780102    2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 17:56:42.780199 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.780137    2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 17:56:42.780239 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.780217    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 17:56:42.780475 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.780245    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 17:56:42.780475 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.780277    2555 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 17:56:42.781224 ip-10-0-140-23 systemd[1]: Started Kubernetes Kubelet.
Jul 17 17:56:42.783145 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.783123    2555 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 17:56:42.783233 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.783165    2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 17:56:42.785548 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.785523    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rfbb2"
Jul 17 17:56:42.786774 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.786756    2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 17:56:42.790110 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.790089    2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 17:56:42.790235 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.790109    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 17:56:42.790764 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.790659    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:42.790860 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.790834    2555 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 17:56:42.790860 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.790851    2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 17:56:42.790957 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.790854    2555 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 17:56:42.791122 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791104    2555 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 17:56:42.791122 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791122    2555 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 17:56:42.791355 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.790405    2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-23.ec2.internal.18c32567417c4276  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-23.ec2.internal,UID:ip-10-0-140-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-23.ec2.internal,},FirstTimestamp:2026-07-17 17:56:42.779370102 +0000 UTC m=+0.364846355,LastTimestamp:2026-07-17 17:56:42.779370102 +0000 UTC m=+0.364846355,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-23.ec2.internal,}"
Jul 17 17:56:42.791521 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791507    2555 factory.go:153] Registering CRI-O factory
Jul 17 17:56:42.791574 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791526    2555 factory.go:223] Registration of the crio container factory successfully
Jul 17 17:56:42.791623 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791579    2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 17:56:42.791623 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791590    2555 factory.go:55] Registering systemd factory
Jul 17 17:56:42.791623 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791598    2555 factory.go:223] Registration of the systemd container factory successfully
Jul 17 17:56:42.791623 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791621    2555 factory.go:103] Registering Raw factory
Jul 17 17:56:42.791873 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.791639    2555 manager.go:1196] Started watching for new ooms in manager
Jul 17 17:56:42.792282 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.792264    2555 manager.go:349] Starting recovery of all containers
Jul 17 17:56:42.795187 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.795159    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rfbb2"
Jul 17 17:56:42.800638 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.800607    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 17:56:42.800911 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.800884    2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 17:56:42.803929 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.803776    2555 manager.go:354] Recovery completed
Jul 17 17:56:42.808486 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.808472    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:56:42.810523 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.810507    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:56:42.810618 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.810541    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:56:42.810618 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.810557    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:56:42.811233 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.811217    2555 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 17:56:42.811233 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.811226    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:56:42.811370 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.811230    2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 17:56:42.811370 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.811281    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 17:56:42.812008 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.811992    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:56:42.812057 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.812020    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:56:42.812057 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.812031    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:56:42.812391 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.812379    2555 policy_none.go:49] "None policy: Start"
Jul 17 17:56:42.812428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.812396    2555 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 17:56:42.812428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.812407    2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 17:56:42.813427 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.813411    2555 policy_none.go:47] "Start"
Jul 17 17:56:42.850378 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.850358    2555 manager.go:337] "Starting Device Plugin manager"
Jul 17 17:56:42.850527 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.850407    2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 17:56:42.850527 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.850423    2555 server.go:87] "Starting device plugin registration server"
Jul 17 17:56:42.850786 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.850768    2555 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 17:56:42.850899 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.850785    2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 17:56:42.850899 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.850884    2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 17:56:42.851009 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.850980    2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 17:56:42.851009 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.850990    2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 17:56:42.851607 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.851579    2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 17:56:42.851710 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.851649    2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:42.935578 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.935532    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 17:56:42.936789 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.936771    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 17:56:42.936948 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.936815    2555 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 17:56:42.936948 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.936845    2555 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 17:56:42.936948 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.936888    2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 17:56:42.939302 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.939278    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:56:42.951473 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.951409    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:56:42.952372 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.952356    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:56:42.952439 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.952391    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:56:42.952439 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.952404    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:56:42.952439 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.952430    2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-23.ec2.internal"
Jul 17 17:56:42.959738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:42.959706    2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-23.ec2.internal"
Jul 17 17:56:42.959799 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.959745    2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-23.ec2.internal\": node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:42.975999 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:42.975974    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.037747 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.037699    2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"]
Jul 17 17:56:43.037914 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.037789    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:56:43.038937 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.038911    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:56:43.039064 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.038949    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:56:43.039064 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.038964    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:56:43.040178 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.040161    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:56:43.040351 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.040336    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.040410 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.040375    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:56:43.041018 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.041006    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:56:43.041084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.041021    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:56:43.041084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.041033    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:56:43.041084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.041044    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:56:43.041084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.041050    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:56:43.041084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.041055    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:56:43.042432 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.042411    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.042490 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.042457    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 17:56:43.043401 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.043368    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 17:56:43.043515 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.043418    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 17:56:43.043515 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.043435    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Jul 17 17:56:43.060527 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.060492    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-23.ec2.internal\" not found" node="ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.064261 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.064239    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-23.ec2.internal\" not found" node="ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.076459 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.076430    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.093346 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.093311    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/21e86a514ca1a2823ff3a56bf309c8b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"21e86a514ca1a2823ff3a56bf309c8b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.093346 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.093347    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e86a514ca1a2823ff3a56bf309c8b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"21e86a514ca1a2823ff3a56bf309c8b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.093521 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.093378    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9e7f913e9ca157cda0a9d37dab0aefd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-23.ec2.internal\" (UID: \"e9e7f913e9ca157cda0a9d37dab0aefd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.176724 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.176685    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.194424 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.194395    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/21e86a514ca1a2823ff3a56bf309c8b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"21e86a514ca1a2823ff3a56bf309c8b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.194525 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.194431    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e86a514ca1a2823ff3a56bf309c8b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"21e86a514ca1a2823ff3a56bf309c8b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.194525 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.194450    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9e7f913e9ca157cda0a9d37dab0aefd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-23.ec2.internal\" (UID: \"e9e7f913e9ca157cda0a9d37dab0aefd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.194525 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.194498    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9e7f913e9ca157cda0a9d37dab0aefd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-23.ec2.internal\" (UID: \"e9e7f913e9ca157cda0a9d37dab0aefd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.194525 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.194512    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/21e86a514ca1a2823ff3a56bf309c8b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"21e86a514ca1a2823ff3a56bf309c8b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.194654 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.194531    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e86a514ca1a2823ff3a56bf309c8b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"21e86a514ca1a2823ff3a56bf309c8b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.277414 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.277307    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.366308 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.366264    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.367379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.367357    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:43.377669 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.377630    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.482369 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.482318    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.586397 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.586312    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.689795 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.689751    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.724654 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.724599    2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 17:56:43.725085 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.724767    2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 17:56:43.790226 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.790191    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.798947 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.798870    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 17:51:43 +0000 UTC" deadline="2028-02-18 20:13:09.58454793 +0000 UTC"
Jul 17 17:56:43.799037 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.798945    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13946h16m25.785607222s"
Jul 17 17:56:43.893559 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.893467    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:43.912825 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:43.912805    2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:56:43.994473 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:43.994440    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:44.088947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:44.088913    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e86a514ca1a2823ff3a56bf309c8b5.slice/crio-b6417d3073d73e675f14841a03a74809403fa39e7f36713f7a2076d3eb09234f WatchSource:0}: Error finding container b6417d3073d73e675f14841a03a74809403fa39e7f36713f7a2076d3eb09234f: Status 404 returned error can't find the container with id b6417d3073d73e675f14841a03a74809403fa39e7f36713f7a2076d3eb09234f
Jul 17 17:56:44.089476 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:44.089459    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e7f913e9ca157cda0a9d37dab0aefd.slice/crio-1660f29492bdabd92a45205ed3c1b55f533d650cb4c0ff32492588891359df52 WatchSource:0}: Error finding container 1660f29492bdabd92a45205ed3c1b55f533d650cb4c0ff32492588891359df52: Status 404 returned error can't find the container with id 1660f29492bdabd92a45205ed3c1b55f533d650cb4c0ff32492588891359df52
Jul 17 17:56:44.094916 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.094892    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:44.095022 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.095008    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 17:56:44.196816 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.196707    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:44.297282 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.297243    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Jul 17 17:56:44.322082 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.322057    2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:56:44.375537 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.375503    2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:56:44.390765 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.390734    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:44.400471 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.400434    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 17:56:44.402913 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.402891    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Jul 17 17:56:44.513915 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.513833    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 17:56:44.770708 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.770519    2555 apiserver.go:52] "Watching apiserver"
Jul 17 17:56:44.777493 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.777393    2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 17:56:44.777985 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.777946    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-k88rf","openshift-multus/network-metrics-daemon-5n67b","openshift-network-diagnostics/network-check-target-c9xd4","openshift-ovn-kubernetes/ovnkube-node-pm5c6","kube-system/global-pull-secret-syncer-snwmp","kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2","openshift-dns/node-resolver-t6frw","openshift-multus/multus-additional-cni-plugins-v8q9v","openshift-network-operator/iptables-alerter-m8f7l","kube-system/konnectivity-agent-wbb86","openshift-cluster-node-tuning-operator/tuned-9llkd","openshift-image-registry/node-ca-d4bph","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"]
Jul 17 17:56:44.780936 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.780910    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.783520 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.783493    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.784833 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.784809    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 17:56:44.784944 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.784829    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gfvzt\""
Jul 17 17:56:44.784944 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.784819    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 17:56:44.785909 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.785891    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:44.786019 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.785964    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:44.786144 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.786121    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 17:56:44.786561 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.786538    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 17:56:44.786648 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.786568    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 17:56:44.786883 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.786861    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 17:56:44.786994 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.786873    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nk4pk\""
Jul 17 17:56:44.788594 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.788570    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.790344 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.790322    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 17:56:44.791125 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.791039    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.791125 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.791109    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:44.793540 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.793522    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.796275 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.796254    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.797442 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.797419    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 17:56:44.797664 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.797633    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 17:56:44.797746 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.797685    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 17:56:44.797746 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.797697    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 17:56:44.797746 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.797740    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 17:56:44.797964 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.797945    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 17:56:44.798059 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.798042    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vkgj2\""
Jul 17 17:56:44.798121 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.798069    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 17:56:44.798121 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.798074    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 17:56:44.798597 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.798442    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-756sv\""
Jul 17 17:56:44.798865 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.798846    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 17:56:44.804666 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804621    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.804803 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804668    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c868384-1df1-438e-8e7c-6a7d578a58af-kubelet-config\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.804803 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804704    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.804803 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804759    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-tmp-dir\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.804803 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804796    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt5m2\" (UniqueName: \"kubernetes.io/projected/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-kube-api-access-bt5m2\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.805024 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804832    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.805024 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804873    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5736d32-71da-4d48-8c66-fdfd0301ea1f-host\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.805024 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804905    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovn-node-metrics-cert\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805024 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804937    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-systemd-units\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805024 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804968    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-slash\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805024 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.804996    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-ovn\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805024 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805019    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c868384-1df1-438e-8e7c-6a7d578a58af-dbus\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805048    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805089    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-sys-fs\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805118    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-systemd\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805156    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-cni-netd\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805180    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-env-overrides\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805212    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovnkube-script-lib\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805240    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5736d32-71da-4d48-8c66-fdfd0301ea1f-serviceca\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805285    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-run-netns\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805315    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-socket-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.805379 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805359    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-etc-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805891 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805394    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805891 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805424    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-kubelet\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805891 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805455    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-var-lib-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805891 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805599    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-run-ovn-kubernetes\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805891 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805644    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-cni-bin\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.805891 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805820    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:44.805891 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805860    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.806229 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.805916    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:44.806229 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805707    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-registration-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.806229 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.805992    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-device-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807412    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n6ww\" (UniqueName: \"kubernetes.io/projected/1427b65f-7603-4dd8-9e30-d26adac520d5-kube-api-access-5n6ww\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807464    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkp2g\" (UniqueName: \"kubernetes.io/projected/f5736d32-71da-4d48-8c66-fdfd0301ea1f-kube-api-access-hkp2g\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807498    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807533    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-node-log\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807565    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-log-socket\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807597    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovnkube-config\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807629    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt4zq\" (UniqueName: \"kubernetes.io/projected/04f7b734-c54b-457d-ba62-ec6cf5ef10af-kube-api-access-dt4zq\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.807801 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.807661    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-hosts-file\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.808773 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.808750    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:44.811787 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.811589    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:44.813440 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.813415    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 17:56:44.813622 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.813595    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 17:56:44.813745 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.813726    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 17:56:44.814018 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.814000    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:56:44.814105 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.814050    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7frs5\""
Jul 17 17:56:44.814775 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.814751    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 17:56:44.815223 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.814933    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9lzs9\""
Jul 17 17:56:44.815312 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.815257    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 17:56:44.815364 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.815339    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d48g6\""
Jul 17 17:56:44.815481 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.815460    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 17:56:44.815633 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.815608    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-csght\""
Jul 17 17:56:44.815747 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.815683    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.816172 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.816153    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 17:56:44.816530 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.816505    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 17:56:44.816789 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.816679    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 17:56:44.818201 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.818176    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 17:56:44.819110 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.819089    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 17:56:44.820706 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.819354    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 17:56:44.820706 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.819665    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4q6d7\""
Jul 17 17:56:44.867606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.867579    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-snktm"
Jul 17 17:56:44.876045 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.876011    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-snktm"
Jul 17 17:56:44.892258 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.892223    2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 17:56:44.907855 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.907815    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-socket-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.908047 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.907873    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-host\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.908047 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.907902    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-os-release\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.908047 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.907942    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-cni-bin\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.908047 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.907971    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-systemd\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.908047 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.907995    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61fd3cf4-956c-45d3-a926-c0b39369e21c-agent-certs\") pod \"konnectivity-agent-wbb86\" (UID: \"61fd3cf4-956c-45d3-a926-c0b39369e21c\") " pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:44.908047 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908022    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-var-lib-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908049    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-run-ovn-kubernetes\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908075    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-registration-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908103    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n6ww\" (UniqueName: \"kubernetes.io/projected/1427b65f-7603-4dd8-9e30-d26adac520d5-kube-api-access-5n6ww\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908138    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkp2g\" (UniqueName: \"kubernetes.io/projected/f5736d32-71da-4d48-8c66-fdfd0301ea1f-kube-api-access-hkp2g\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908166    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-host-slash\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908194    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-kubelet\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908274    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-var-lib-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908325    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-run-ovn-kubernetes\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908338    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-node-log\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908384    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt4zq\" (UniqueName: \"kubernetes.io/projected/04f7b734-c54b-457d-ba62-ec6cf5ef10af-kube-api-access-dt4zq\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908416    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-hosts-file\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908448    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-k8s-cni-cncf-io\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908511    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-etc-kubernetes\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908538    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysconfig\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908572    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-lib-modules\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908603    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-464gn\" (UniqueName: \"kubernetes.io/projected/4a9582ca-6612-4155-9ffc-d9af164ba031-kube-api-access-464gn\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908647    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-hosts-file\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908545    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-node-log\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908756    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908757    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-socket-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908805    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-registration-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908874    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.908921 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908909    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.909644 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908953    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5736d32-71da-4d48-8c66-fdfd0301ea1f-host\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.909644 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.908985    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-system-cni-dir\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.909644 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909040    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl44f\" (UniqueName: \"kubernetes.io/projected/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-kube-api-access-wl44f\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:44.909644 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909069    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wng8n\" (UniqueName: \"kubernetes.io/projected/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-kube-api-access-wng8n\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:44.909644 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909072    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-etc-selinux\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.909644 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909142    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5736d32-71da-4d48-8c66-fdfd0301ea1f-host\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.909644 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909097    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysctl-conf\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909668    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovn-node-metrics-cert\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909700    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909743    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a9582ca-6612-4155-9ffc-d9af164ba031-tmp\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909770    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-slash\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909794    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-ovn\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909819    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c868384-1df1-438e-8e7c-6a7d578a58af-dbus\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909845    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-sys-fs\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909870    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-netns\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909895    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-kubernetes\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909918    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-systemd\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909944    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-env-overrides\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.909966    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5736d32-71da-4d48-8c66-fdfd0301ea1f-serviceca\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910001    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cnibin\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.910030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910025    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-system-cni-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910049    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbbxx\" (UniqueName: \"kubernetes.io/projected/9991312a-7e80-4881-85d1-56d713fe9a8f-kube-api-access-dbbxx\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910071    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-sys\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910102    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-run-netns\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910130    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9991312a-7e80-4881-85d1-56d713fe9a8f-cni-binary-copy\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910153    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-daemon-config\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910178    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-multus-certs\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910202    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-run\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910225    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-etc-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910256    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910282    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910309    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-kubelet\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910333    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-cni-bin\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910358    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-device-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910401    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-conf-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910432    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910508    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.910738 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910515    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-run-netns\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910553    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-systemd\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910564    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-run-ovn\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910620    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-log-socket\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910646    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovnkube-config\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910698    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910730    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b526\" (UniqueName: \"kubernetes.io/projected/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-kube-api-access-4b526\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910754    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-iptables-alerter-script\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910783    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-cni-multus\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910806    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-kubelet\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910827    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c868384-1df1-438e-8e7c-6a7d578a58af-kubelet-config\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910852    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910845    2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910877    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-tmp-dir\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910902    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt5m2\" (UniqueName: \"kubernetes.io/projected/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-kube-api-access-bt5m2\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910927    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910947    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0c868384-1df1-438e-8e7c-6a7d578a58af-dbus\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.911586 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910968    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-log-socket\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.910986    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-cni-bin\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911000    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-sys-fs\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911043    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-etc-openvswitch\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911371    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5736d32-71da-4d48-8c66-fdfd0301ea1f-serviceca\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911423    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-slash\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911467    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-env-overrides\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.911536    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911541    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovnkube-config\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911551    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-device-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911586    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-hostroot\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911596    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0c868384-1df1-438e-8e7c-6a7d578a58af-kubelet-config\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.911630    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret podName:0c868384-1df1-438e-8e7c-6a7d578a58af nodeName:}" failed. No retries permitted until 2026-07-17 17:56:45.411589409 +0000 UTC m=+2.997065645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret") pod "global-pull-secret-syncer-snwmp" (UID: "0c868384-1df1-438e-8e7c-6a7d578a58af") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911854    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-tmp-dir\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911865    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysctl-d\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911891    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-tuned\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911963    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:44.912387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.911991    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-modprobe-d\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912014    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-cni-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912042    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-systemd-units\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912082    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912110    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-systemd-units\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912130    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61fd3cf4-956c-45d3-a926-c0b39369e21c-konnectivity-ca\") pod \"konnectivity-agent-wbb86\" (UID: \"61fd3cf4-956c-45d3-a926-c0b39369e21c\") " pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912176    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1427b65f-7603-4dd8-9e30-d26adac520d5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912215    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-socket-dir-parent\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912246    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-cni-netd\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912272    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovnkube-script-lib\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912298    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-cnibin\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912321    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-os-release\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912329    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04f7b734-c54b-457d-ba62-ec6cf5ef10af-host-cni-netd\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.913260 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.912344    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-var-lib-kubelet\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:44.914074 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.914032    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovnkube-script-lib\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.914935 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.914900    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04f7b734-c54b-457d-ba62-ec6cf5ef10af-ovn-node-metrics-cert\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.916810 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.916787    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:56:44.916932 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.916824    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:56:44.916932 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.916841    2555 projected.go:196] Error preparing data for projected volume kube-api-access-6w2nd for pod openshift-network-diagnostics/network-check-target-c9xd4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:44.917053 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:44.916955    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd podName:1031a638-0acf-4023-91f8-ba906ca32530 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:45.416934237 +0000 UTC m=+3.002410488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6w2nd" (UniqueName: "kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd") pod "network-check-target-c9xd4" (UID: "1031a638-0acf-4023-91f8-ba906ca32530") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:44.919343 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.919318    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkp2g\" (UniqueName: \"kubernetes.io/projected/f5736d32-71da-4d48-8c66-fdfd0301ea1f-kube-api-access-hkp2g\") pod \"node-ca-d4bph\" (UID: \"f5736d32-71da-4d48-8c66-fdfd0301ea1f\") " pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:44.921670 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.921580    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n6ww\" (UniqueName: \"kubernetes.io/projected/1427b65f-7603-4dd8-9e30-d26adac520d5-kube-api-access-5n6ww\") pod \"aws-ebs-csi-driver-node-6wtf2\" (UID: \"1427b65f-7603-4dd8-9e30-d26adac520d5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:44.922597 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.922572    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt4zq\" (UniqueName: \"kubernetes.io/projected/04f7b734-c54b-457d-ba62-ec6cf5ef10af-kube-api-access-dt4zq\") pod \"ovnkube-node-pm5c6\" (UID: \"04f7b734-c54b-457d-ba62-ec6cf5ef10af\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:44.924807 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.924782    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt5m2\" (UniqueName: \"kubernetes.io/projected/44502548-11a1-4a3e-8a4b-07a4f1d5cbb3-kube-api-access-bt5m2\") pod \"node-resolver-t6frw\" (UID: \"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3\") " pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:44.943095 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.942931    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" event={"ID":"21e86a514ca1a2823ff3a56bf309c8b5","Type":"ContainerStarted","Data":"b6417d3073d73e675f14841a03a74809403fa39e7f36713f7a2076d3eb09234f"}
Jul 17 17:56:44.948893 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:44.948854    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal" event={"ID":"e9e7f913e9ca157cda0a9d37dab0aefd","Type":"ContainerStarted","Data":"1660f29492bdabd92a45205ed3c1b55f533d650cb4c0ff32492588891359df52"}
Jul 17 17:56:45.013025 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.012989    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-host-slash\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:45.013025 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013036    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-kubelet\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013065    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-k8s-cni-cncf-io\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013092    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-etc-kubernetes\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013120    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysconfig\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013120    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-host-slash\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013143    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-lib-modules\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013167    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-464gn\" (UniqueName: \"kubernetes.io/projected/4a9582ca-6612-4155-9ffc-d9af164ba031-kube-api-access-464gn\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013189    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-k8s-cni-cncf-io\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013210    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-etc-kubernetes\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013199    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-system-cni-dir\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013130    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-kubelet\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013245    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-system-cni-dir\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.013280 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013275    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl44f\" (UniqueName: \"kubernetes.io/projected/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-kube-api-access-wl44f\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013304    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wng8n\" (UniqueName: \"kubernetes.io/projected/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-kube-api-access-wng8n\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013314    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysconfig\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013336    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysctl-conf\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013360    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-lib-modules\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013385    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013511    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysctl-conf\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013557    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a9582ca-6612-4155-9ffc-d9af164ba031-tmp\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013706    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-netns\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013741    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-kubernetes\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013768    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cnibin\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013786    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-netns\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013793    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-system-cni-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013838    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbbxx\" (UniqueName: \"kubernetes.io/projected/9991312a-7e80-4881-85d1-56d713fe9a8f-kube-api-access-dbbxx\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013846    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-system-cni-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013864    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-sys\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013888    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cnibin\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.013907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013904    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-kubernetes\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013889    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9991312a-7e80-4881-85d1-56d713fe9a8f-cni-binary-copy\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013956    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-sys\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.013985    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-daemon-config\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014013    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-multus-certs\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014039    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-run\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014067    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014094    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-conf-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014136    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-run-multus-certs\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014143    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014173    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b526\" (UniqueName: \"kubernetes.io/projected/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-kube-api-access-4b526\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014199    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-iptables-alerter-script\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014226    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-cni-multus\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014237    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-conf-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014269    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014293    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-hostroot\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014312    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.014732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014317    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysctl-d\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014343    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-tuned\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014361    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-cni-multus\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014370    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014417    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-modprobe-d\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014445    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-cni-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014449    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9991312a-7e80-4881-85d1-56d713fe9a8f-cni-binary-copy\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014473    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61fd3cf4-956c-45d3-a926-c0b39369e21c-konnectivity-ca\") pod \"konnectivity-agent-wbb86\" (UID: \"61fd3cf4-956c-45d3-a926-c0b39369e21c\") " pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.014320    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014501    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-socket-dir-parent\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.014539    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs podName:b6b5f8c0-20a0-4d96-9ab4-b0e256242982 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:45.514518921 +0000 UTC m=+3.099995157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs") pod "network-metrics-daemon-5n67b" (UID: "b6b5f8c0-20a0-4d96-9ab4-b0e256242982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014561    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-cnibin\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014571    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-socket-dir-parent\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014605    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-hostroot\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014627    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-cnibin\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014647    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-os-release\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014652    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-daemon-config\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014674    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-var-lib-kubelet\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.015606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014734    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-host\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014788    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-sysctl-d\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014782    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-os-release\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014834    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-cni-bin\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014842    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014857    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-os-release\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014863    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-systemd\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014912    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-var-lib-kubelet\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014917    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-os-release\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014963    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-host-var-lib-cni-bin\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014857    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.014966    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-systemd\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015028    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-host\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015036    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61fd3cf4-956c-45d3-a926-c0b39369e21c-agent-certs\") pod \"konnectivity-agent-wbb86\" (UID: \"61fd3cf4-956c-45d3-a926-c0b39369e21c\") " pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015134    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-modprobe-d\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015134    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9991312a-7e80-4881-85d1-56d713fe9a8f-multus-cni-dir\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015324    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.016606 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015386    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a9582ca-6612-4155-9ffc-d9af164ba031-run\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.017481 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015740    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-iptables-alerter-script\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:45.017481 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.015754    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61fd3cf4-956c-45d3-a926-c0b39369e21c-konnectivity-ca\") pod \"konnectivity-agent-wbb86\" (UID: \"61fd3cf4-956c-45d3-a926-c0b39369e21c\") " pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:45.017481 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.017036    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a9582ca-6612-4155-9ffc-d9af164ba031-tmp\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.018310 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.018245    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61fd3cf4-956c-45d3-a926-c0b39369e21c-agent-certs\") pod \"konnectivity-agent-wbb86\" (UID: \"61fd3cf4-956c-45d3-a926-c0b39369e21c\") " pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:45.018729 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.018706    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a9582ca-6612-4155-9ffc-d9af164ba031-etc-tuned\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.023122 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.023044    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl44f\" (UniqueName: \"kubernetes.io/projected/3da3d465-c93a-45fc-83a4-5a7fee8a4f4a-kube-api-access-wl44f\") pod \"iptables-alerter-m8f7l\" (UID: \"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a\") " pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:45.024096 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.024046    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-464gn\" (UniqueName: \"kubernetes.io/projected/4a9582ca-6612-4155-9ffc-d9af164ba031-kube-api-access-464gn\") pod \"tuned-9llkd\" (UID: \"4a9582ca-6612-4155-9ffc-d9af164ba031\") " pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.024481 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.024463    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbbxx\" (UniqueName: \"kubernetes.io/projected/9991312a-7e80-4881-85d1-56d713fe9a8f-kube-api-access-dbbxx\") pod \"multus-k88rf\" (UID: \"9991312a-7e80-4881-85d1-56d713fe9a8f\") " pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.024865 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.024842    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b526\" (UniqueName: \"kubernetes.io/projected/8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1-kube-api-access-4b526\") pod \"multus-additional-cni-plugins-v8q9v\" (UID: \"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1\") " pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.025185 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.025152    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wng8n\" (UniqueName: \"kubernetes.io/projected/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-kube-api-access-wng8n\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:45.049766 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.049717    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 17:56:45.093732 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.093690    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2"
Jul 17 17:56:45.104604 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.104567    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d4bph"
Jul 17 17:56:45.116751 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.116713    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:56:45.124080 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.124055    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t6frw"
Jul 17 17:56:45.134145 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.134116    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8q9v"
Jul 17 17:56:45.143381 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.143353    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k88rf"
Jul 17 17:56:45.152607 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.152583    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8f7l"
Jul 17 17:56:45.161764 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.161732    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:56:45.168037 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.168003    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9llkd"
Jul 17 17:56:45.418553 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.418469    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:45.418553 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.418524    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:45.419020 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.418851    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:45.419132 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.419113    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret podName:0c868384-1df1-438e-8e7c-6a7d578a58af nodeName:}" failed. No retries permitted until 2026-07-17 17:56:46.419075579 +0000 UTC m=+4.004551827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret") pod "global-pull-secret-syncer-snwmp" (UID: "0c868384-1df1-438e-8e7c-6a7d578a58af") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:45.419132 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.419120    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:56:45.419344 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.419152    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:56:45.419344 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.419177    2555 projected.go:196] Error preparing data for projected volume kube-api-access-6w2nd for pod openshift-network-diagnostics/network-check-target-c9xd4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:45.419344 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.419248    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd podName:1031a638-0acf-4023-91f8-ba906ca32530 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:46.419224568 +0000 UTC m=+4.004700819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6w2nd" (UniqueName: "kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd") pod "network-check-target-c9xd4" (UID: "1031a638-0acf-4023-91f8-ba906ca32530") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:45.519612 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.519565    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:45.519798 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.519767    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:45.519870 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.519856    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs podName:b6b5f8c0-20a0-4d96-9ab4-b0e256242982 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:46.519830456 +0000 UTC m=+4.105306714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs") pod "network-metrics-daemon-5n67b" (UID: "b6b5f8c0-20a0-4d96-9ab4-b0e256242982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:45.879875 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.879744    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 17:51:45 +0000 UTC" deadline="2027-12-22 18:51:21.637323649 +0000 UTC"
Jul 17 17:56:45.879875 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.879812    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12552h54m35.757515088s"
Jul 17 17:56:45.937757 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.937725    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:45.937928 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.937855    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:45.937928 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:45.937900    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:45.938063 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:45.937989    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:46.130181 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.130113    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f7b734_c54b_457d_ba62_ec6cf5ef10af.slice/crio-1c5509ecfe42fb1a65d96800b3844494c1d1bd4cb0e558ab2983f4c09299dea8 WatchSource:0}: Error finding container 1c5509ecfe42fb1a65d96800b3844494c1d1bd4cb0e558ab2983f4c09299dea8: Status 404 returned error can't find the container with id 1c5509ecfe42fb1a65d96800b3844494c1d1bd4cb0e558ab2983f4c09299dea8
Jul 17 17:56:46.131947 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.131922    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da3d465_c93a_45fc_83a4_5a7fee8a4f4a.slice/crio-abf9c1a3c66b0593e484947621e0bc4f033a41c79df89a9c2194db9a31e045e4 WatchSource:0}: Error finding container abf9c1a3c66b0593e484947621e0bc4f033a41c79df89a9c2194db9a31e045e4: Status 404 returned error can't find the container with id abf9c1a3c66b0593e484947621e0bc4f033a41c79df89a9c2194db9a31e045e4
Jul 17 17:56:46.134285 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.134263    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44502548_11a1_4a3e_8a4b_07a4f1d5cbb3.slice/crio-fb3723a36bbe1d326de4777d9dc160f31d3ddc097c1fc29fe9651eb111012dbb WatchSource:0}: Error finding container fb3723a36bbe1d326de4777d9dc160f31d3ddc097c1fc29fe9651eb111012dbb: Status 404 returned error can't find the container with id fb3723a36bbe1d326de4777d9dc160f31d3ddc097c1fc29fe9651eb111012dbb
Jul 17 17:56:46.135015 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.134986    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5736d32_71da_4d48_8c66_fdfd0301ea1f.slice/crio-97cec47691dd6dff557065ed734c90660dcf64d306bf078e73caa8bd17cd54c4 WatchSource:0}: Error finding container 97cec47691dd6dff557065ed734c90660dcf64d306bf078e73caa8bd17cd54c4: Status 404 returned error can't find the container with id 97cec47691dd6dff557065ed734c90660dcf64d306bf078e73caa8bd17cd54c4
Jul 17 17:56:46.136384 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.135957    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fd3cf4_956c_45d3_a926_c0b39369e21c.slice/crio-75194b20a18d2616ffa4f6b4cbc13a1fe88c855df464b4022b147cde94e82c72 WatchSource:0}: Error finding container 75194b20a18d2616ffa4f6b4cbc13a1fe88c855df464b4022b147cde94e82c72: Status 404 returned error can't find the container with id 75194b20a18d2616ffa4f6b4cbc13a1fe88c855df464b4022b147cde94e82c72
Jul 17 17:56:46.138488 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.138436    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1427b65f_7603_4dd8_9e30_d26adac520d5.slice/crio-6ee61bc4115369794f0e7246f0b0f983cca6f5b07bd967361e092ecbd3414427 WatchSource:0}: Error finding container 6ee61bc4115369794f0e7246f0b0f983cca6f5b07bd967361e092ecbd3414427: Status 404 returned error can't find the container with id 6ee61bc4115369794f0e7246f0b0f983cca6f5b07bd967361e092ecbd3414427
Jul 17 17:56:46.139815 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.139511    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9a49e5_e8c3_4fcf_8517_a6024e41f7a1.slice/crio-a2cfc0aee7704e8bd22796266dc75af5908784099933852b3181226925bb8029 WatchSource:0}: Error finding container a2cfc0aee7704e8bd22796266dc75af5908784099933852b3181226925bb8029: Status 404 returned error can't find the container with id a2cfc0aee7704e8bd22796266dc75af5908784099933852b3181226925bb8029
Jul 17 17:56:46.140610 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:56:46.140299    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9991312a_7e80_4881_85d1_56d713fe9a8f.slice/crio-123fc76326ceba9b05df762beeba3624558ee3d4e35d31507b7315f69529aab2 WatchSource:0}: Error finding container 123fc76326ceba9b05df762beeba3624558ee3d4e35d31507b7315f69529aab2: Status 404 returned error can't find the container with id 123fc76326ceba9b05df762beeba3624558ee3d4e35d31507b7315f69529aab2
Jul 17 17:56:46.428873 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.428504    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:46.428873 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.428837    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:46.428873 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.428695    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:56:46.429098 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.428885    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:56:46.429098 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.428898    2555 projected.go:196] Error preparing data for projected volume kube-api-access-6w2nd for pod openshift-network-diagnostics/network-check-target-c9xd4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:46.429098 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.428959    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd podName:1031a638-0acf-4023-91f8-ba906ca32530 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:48.428943509 +0000 UTC m=+6.014419742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6w2nd" (UniqueName: "kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd") pod "network-check-target-c9xd4" (UID: "1031a638-0acf-4023-91f8-ba906ca32530") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:46.429098 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.428979    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:46.429098 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.429042    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret podName:0c868384-1df1-438e-8e7c-6a7d578a58af nodeName:}" failed. No retries permitted until 2026-07-17 17:56:48.4290228 +0000 UTC m=+6.014499033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret") pod "global-pull-secret-syncer-snwmp" (UID: "0c868384-1df1-438e-8e7c-6a7d578a58af") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:46.529374 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.529331    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:46.529554 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.529497    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:46.529602 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.529562    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs podName:b6b5f8c0-20a0-4d96-9ab4-b0e256242982 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:48.529545494 +0000 UTC m=+6.115021739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs") pod "network-metrics-daemon-5n67b" (UID: "b6b5f8c0-20a0-4d96-9ab4-b0e256242982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:46.880424 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.880374    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 17:51:45 +0000 UTC" deadline="2028-01-20 15:16:51.076158679 +0000 UTC"
Jul 17 17:56:46.880424 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.880422    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13245h20m4.195741568s"
Jul 17 17:56:46.940133 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.940099    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:46.940324 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:46.940249    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:46.967347 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.967240    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2" event={"ID":"1427b65f-7603-4dd8-9e30-d26adac520d5","Type":"ContainerStarted","Data":"6ee61bc4115369794f0e7246f0b0f983cca6f5b07bd967361e092ecbd3414427"}
Jul 17 17:56:46.972914 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.972869    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t6frw" event={"ID":"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3","Type":"ContainerStarted","Data":"fb3723a36bbe1d326de4777d9dc160f31d3ddc097c1fc29fe9651eb111012dbb"}
Jul 17 17:56:46.988300 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.988261    2555 generic.go:356] "Generic (PLEG): container finished" podID="21e86a514ca1a2823ff3a56bf309c8b5" containerID="53e2c45cd0207fe89cd15a19fda3efe1122d799bb7662b8694a5798e255bf1dc" exitCode=0
Jul 17 17:56:46.988671 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.988597    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" event={"ID":"21e86a514ca1a2823ff3a56bf309c8b5","Type":"ContainerDied","Data":"53e2c45cd0207fe89cd15a19fda3efe1122d799bb7662b8694a5798e255bf1dc"}
Jul 17 17:56:46.999579 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:46.999544    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal" event={"ID":"e9e7f913e9ca157cda0a9d37dab0aefd","Type":"ContainerStarted","Data":"dde5906de4cbf70ac794f3ad5db656bc36191938bd62f946e0d5de253292d213"}
Jul 17 17:56:47.015049 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.015009    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d4bph" event={"ID":"f5736d32-71da-4d48-8c66-fdfd0301ea1f","Type":"ContainerStarted","Data":"97cec47691dd6dff557065ed734c90660dcf64d306bf078e73caa8bd17cd54c4"}
Jul 17 17:56:47.025132 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.025071    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal" podStartSLOduration=3.025044559 podStartE2EDuration="3.025044559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:56:47.024937763 +0000 UTC m=+4.610414020" watchObservedRunningTime="2026-07-17 17:56:47.025044559 +0000 UTC m=+4.610520875"
Jul 17 17:56:47.032623 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.032539    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wbb86" event={"ID":"61fd3cf4-956c-45d3-a926-c0b39369e21c","Type":"ContainerStarted","Data":"75194b20a18d2616ffa4f6b4cbc13a1fe88c855df464b4022b147cde94e82c72"}
Jul 17 17:56:47.042259 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.042155    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8f7l" event={"ID":"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a","Type":"ContainerStarted","Data":"abf9c1a3c66b0593e484947621e0bc4f033a41c79df89a9c2194db9a31e045e4"}
Jul 17 17:56:47.058549 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.058503    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"1c5509ecfe42fb1a65d96800b3844494c1d1bd4cb0e558ab2983f4c09299dea8"}
Jul 17 17:56:47.074473 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.074335    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k88rf" event={"ID":"9991312a-7e80-4881-85d1-56d713fe9a8f","Type":"ContainerStarted","Data":"123fc76326ceba9b05df762beeba3624558ee3d4e35d31507b7315f69529aab2"}
Jul 17 17:56:47.095525 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.095410    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9llkd" event={"ID":"4a9582ca-6612-4155-9ffc-d9af164ba031","Type":"ContainerStarted","Data":"02ad37f1857056e9fda35006da2b00cb627a972b7194b4b0ab360d4e4dcd6d9d"}
Jul 17 17:56:47.104598 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.104551    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerStarted","Data":"a2cfc0aee7704e8bd22796266dc75af5908784099933852b3181226925bb8029"}
Jul 17 17:56:47.937326 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.937284    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:47.937886 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:47.937445    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:47.937971 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:47.937931    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:47.938050 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:47.938027    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:48.127425 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:48.127349    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" event={"ID":"21e86a514ca1a2823ff3a56bf309c8b5","Type":"ContainerStarted","Data":"fdd02ed112193ae6a83e2994d6ea0079f9c198a7f37506a1c9a86e5951884797"}
Jul 17 17:56:48.448016 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:48.447973    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:48.448198 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:48.448036    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:48.448274 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.448186    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:48.448329 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.448305    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret podName:0c868384-1df1-438e-8e7c-6a7d578a58af nodeName:}" failed. No retries permitted until 2026-07-17 17:56:52.4482825 +0000 UTC m=+10.033758757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret") pod "global-pull-secret-syncer-snwmp" (UID: "0c868384-1df1-438e-8e7c-6a7d578a58af") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:48.448983 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.448961    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:56:48.449097 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.448992    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:56:48.449097 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.449006    2555 projected.go:196] Error preparing data for projected volume kube-api-access-6w2nd for pod openshift-network-diagnostics/network-check-target-c9xd4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:48.449097 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.449053    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd podName:1031a638-0acf-4023-91f8-ba906ca32530 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:52.449035778 +0000 UTC m=+10.034512017 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6w2nd" (UniqueName: "kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd") pod "network-check-target-c9xd4" (UID: "1031a638-0acf-4023-91f8-ba906ca32530") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:48.549898 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:48.549260    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:48.549898 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.549457    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:48.549898 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.549523    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs podName:b6b5f8c0-20a0-4d96-9ab4-b0e256242982 nodeName:}" failed. No retries permitted until 2026-07-17 17:56:52.549504221 +0000 UTC m=+10.134980476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs") pod "network-metrics-daemon-5n67b" (UID: "b6b5f8c0-20a0-4d96-9ab4-b0e256242982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:48.941368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:48.940623    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:48.941368 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:48.940806    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:49.938030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:49.937989    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:49.938217 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:49.938139    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:49.938625 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:49.938605    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:49.938747 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:49.938714    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:50.939688 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:50.937683    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:50.939688 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:50.937832    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:51.937596 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:51.937557    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:51.937784 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:51.937709    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:51.938072 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:51.938049    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:51.938187 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:51.938165    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:52.483956 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:52.483913    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:52.484406 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:52.484039    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:52.484406 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.484079    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:52.484406 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.484137    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:56:52.484406 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.484146    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret podName:0c868384-1df1-438e-8e7c-6a7d578a58af nodeName:}" failed. No retries permitted until 2026-07-17 17:57:00.484127881 +0000 UTC m=+18.069604131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret") pod "global-pull-secret-syncer-snwmp" (UID: "0c868384-1df1-438e-8e7c-6a7d578a58af") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:56:52.484406 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.484154    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:56:52.484406 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.484167    2555 projected.go:196] Error preparing data for projected volume kube-api-access-6w2nd for pod openshift-network-diagnostics/network-check-target-c9xd4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:52.484406 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.484214    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd podName:1031a638-0acf-4023-91f8-ba906ca32530 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:00.484202476 +0000 UTC m=+18.069678709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6w2nd" (UniqueName: "kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd") pod "network-check-target-c9xd4" (UID: "1031a638-0acf-4023-91f8-ba906ca32530") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:56:52.585707 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:52.585591    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:52.585883 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.585759    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:52.585949 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.585914    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs podName:b6b5f8c0-20a0-4d96-9ab4-b0e256242982 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:00.585893071 +0000 UTC m=+18.171369300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs") pod "network-metrics-daemon-5n67b" (UID: "b6b5f8c0-20a0-4d96-9ab4-b0e256242982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:56:52.937916 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:52.937829    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:52.938947 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:52.938912    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:53.937086 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:53.937055    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:53.937581 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:53.937180    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:53.937581 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:53.937557    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:53.937710 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:53.937655    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:54.937179 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:54.937145    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:54.937678 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:54.937313    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:55.937851 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:55.937813    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:55.938317 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:55.937870    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:55.938317 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:55.937948    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:55.938317 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:55.938051    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:56.937584 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:56.937552    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:56.937761 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:56.937688    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:57.937957 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:57.937925    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:57.937957 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:57.937954    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:57.938500 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:57.938047    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:57.938500 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:57.938182    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:56:58.937241 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:58.937204    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:56:58.937421 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:58.937344    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:56:59.938006 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:59.937969    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:56:59.938497 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:59.938104    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:56:59.938497 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:56:59.938159    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:56:59.938497 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:56:59.938284    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:00.543223 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:00.543184    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:00.543411 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:00.543239    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:00.543411 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.543331    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:57:00.543411 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.543390    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret podName:0c868384-1df1-438e-8e7c-6a7d578a58af nodeName:}" failed. No retries permitted until 2026-07-17 17:57:16.543371475 +0000 UTC m=+34.128847721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret") pod "global-pull-secret-syncer-snwmp" (UID: "0c868384-1df1-438e-8e7c-6a7d578a58af") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:57:00.543411 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.543333    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:57:00.543628 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.543443    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:57:00.543628 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.543472    2555 projected.go:196] Error preparing data for projected volume kube-api-access-6w2nd for pod openshift-network-diagnostics/network-check-target-c9xd4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:57:00.543628 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.543527    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd podName:1031a638-0acf-4023-91f8-ba906ca32530 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:16.543510711 +0000 UTC m=+34.128986956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6w2nd" (UniqueName: "kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd") pod "network-check-target-c9xd4" (UID: "1031a638-0acf-4023-91f8-ba906ca32530") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:57:00.644461 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:00.644429    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:00.644665 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.644543    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:57:00.644665 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.644610    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs podName:b6b5f8c0-20a0-4d96-9ab4-b0e256242982 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:16.644589615 +0000 UTC m=+34.230065847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs") pod "network-metrics-daemon-5n67b" (UID: "b6b5f8c0-20a0-4d96-9ab4-b0e256242982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:57:00.938050 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:00.937957    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:00.938484 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:00.938109    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:01.938088 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:01.938048    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:01.938588 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:01.938094    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:01.938588 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:01.938184    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:01.938588 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:01.938329    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:02.937838 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:02.937800    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:02.937996 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:02.937905    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:03.937999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:03.937957    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:03.938458 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:03.937958    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:03.938458 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:03.938111    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:03.938458 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:03.938237    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:04.937340 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:04.937294    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:04.937555 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:04.937439    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:05.937907 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:05.937466    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:05.938460 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:05.937496    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:05.938460 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:05.938028    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:05.938460 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:05.938055    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:06.169437 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.169203    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k88rf" event={"ID":"9991312a-7e80-4881-85d1-56d713fe9a8f","Type":"ContainerStarted","Data":"72e1433d1c2152835209301a9ad1c735bb6d166c3eb9fb4f3f4b1e76221f133d"}
Jul 17 17:57:06.171206 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.171180    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9llkd" event={"ID":"4a9582ca-6612-4155-9ffc-d9af164ba031","Type":"ContainerStarted","Data":"74cff1d29796025cbb3866764166fcf0f84bdf1b5dc4885b579b11a37cbf23e5"}
Jul 17 17:57:06.173050 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.172950    2555 generic.go:356] "Generic (PLEG): container finished" podID="8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1" containerID="1ceeedfbdcfe80acb790a62772f4da80191cd41d283afd9ed145cd96bd1464c1" exitCode=0
Jul 17 17:57:06.173050 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.172995    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerDied","Data":"1ceeedfbdcfe80acb790a62772f4da80191cd41d283afd9ed145cd96bd1464c1"}
Jul 17 17:57:06.174966 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.174935    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2" event={"ID":"1427b65f-7603-4dd8-9e30-d26adac520d5","Type":"ContainerStarted","Data":"8b679a7473ce7c0ad28ce78c39c7a7839042cc1cbdfb54774314d29b9fb9db47"}
Jul 17 17:57:06.176701 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.176676    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t6frw" event={"ID":"44502548-11a1-4a3e-8a4b-07a4f1d5cbb3","Type":"ContainerStarted","Data":"0d67beaaad4c5498b300a93a8692991aabfb5b495d63bf4e0db824714aed9b15"}
Jul 17 17:57:06.178924 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.178890    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d4bph" event={"ID":"f5736d32-71da-4d48-8c66-fdfd0301ea1f","Type":"ContainerStarted","Data":"d52c47f73500f83901e2b7d2c109ca98acd352c31bb7b4a8d58644ab92deb710"}
Jul 17 17:57:06.180656 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.180623    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wbb86" event={"ID":"61fd3cf4-956c-45d3-a926-c0b39369e21c","Type":"ContainerStarted","Data":"047bf9246d624e66263a6202a40d61529dd8715696a7a7f43f6aa0ed82f2529d"}
Jul 17 17:57:06.184244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.184220    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"17e5c5afbfcf3bd7a45f5d1a20976abb8e1e678fd05dd0175e135fbe8458f05d"}
Jul 17 17:57:06.184357 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.184279    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"6fc2b83df057242b3b21ec584ec030f3f6051f8d046e8e0e75637934e060d648"}
Jul 17 17:57:06.184357 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.184316    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"75b7db738ed3f76065d1dc5ecd218f776817fb700acbaf469b983c23d3ec876d"}
Jul 17 17:57:06.184472 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.184361    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"7b934d7c4719118446f1c9ea14ac6111b439663d3ece39c0ec73fbd7be0c0226"}
Jul 17 17:57:06.184472 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.184400    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"61af92fd4b12b09bf270d7389eacd7f40d215d3e715927cd8a268f64b949a133"}
Jul 17 17:57:06.184472 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.184435    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"a16dc500420f4e0bff159edfa2d7c2da08297e9f1304ec0d632119ebba3eaff8"}
Jul 17 17:57:06.191512 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.190488    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" podStartSLOduration=22.190470146 podStartE2EDuration="22.190470146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:56:48.14400803 +0000 UTC m=+5.729484288" watchObservedRunningTime="2026-07-17 17:57:06.190470146 +0000 UTC m=+23.775946399"
Jul 17 17:57:06.191512 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.190610    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k88rf" podStartSLOduration=3.974461439 podStartE2EDuration="23.190602823s" totalImagesPullingTime="19.216141384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:06.187070413 +0000 UTC m=+23.772546665" watchObservedRunningTime="2026-07-17 17:57:06.190602823 +0000 UTC m=+23.776079077"
Jul 17 17:57:06.203600 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.203553    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d4bph" podStartSLOduration=4.055672293 podStartE2EDuration="23.203511047s" totalImagesPullingTime="19.147838754s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:06.202909717 +0000 UTC m=+23.788385970" watchObservedRunningTime="2026-07-17 17:57:06.203511047 +0000 UTC m=+23.788987300"
Jul 17 17:57:06.218706 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.218633    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t6frw" podStartSLOduration=4.072154394 podStartE2EDuration="23.218617272s" totalImagesPullingTime="19.146462878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:06.217965837 +0000 UTC m=+23.803442090" watchObservedRunningTime="2026-07-17 17:57:06.218617272 +0000 UTC m=+23.804093523"
Jul 17 17:57:06.255070 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.255020    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wbb86" podStartSLOduration=7.910263417 podStartE2EDuration="23.255004844s" totalImagesPullingTime="15.344741427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:06.254714701 +0000 UTC m=+23.840190963" watchObservedRunningTime="2026-07-17 17:57:06.255004844 +0000 UTC m=+23.840481095"
Jul 17 17:57:06.277445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.277383    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9llkd" podStartSLOduration=4.124424306 podStartE2EDuration="23.277364523s" totalImagesPullingTime="19.152940217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:06.276796637 +0000 UTC m=+23.862272900" watchObservedRunningTime="2026-07-17 17:57:06.277364523 +0000 UTC m=+23.862840777"
Jul 17 17:57:06.912769 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.912729    2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 17:57:06.937894 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.937863    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:06.938354 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:06.937991    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:06.947134 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.947106    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:57:06.947823 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:06.947796    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:57:07.187634 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.187540    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8f7l" event={"ID":"3da3d465-c93a-45fc-83a4-5a7fee8a4f4a","Type":"ContainerStarted","Data":"26babeadaa2606780b9af0068d95971d7ea06e09f0bf1071ef91280e7daa22f7"}
Jul 17 17:57:07.189197 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.189177    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2" event={"ID":"1427b65f-7603-4dd8-9e30-d26adac520d5","Type":"ContainerStarted","Data":"6f7c58938b4f45d9da3c2b5bf0eb682684a08b4b7dd9f3adb2344df746a81470"}
Jul 17 17:57:07.203387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.203335    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m8f7l" podStartSLOduration=5.054169943 podStartE2EDuration="24.203284016s" totalImagesPullingTime="19.149114073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:07.202281388 +0000 UTC m=+24.787757641" watchObservedRunningTime="2026-07-17 17:57:07.203284016 +0000 UTC m=+24.788760268"
Jul 17 17:57:07.866225 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.866117    2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T17:57:06.91275265Z","UUID":"448866c3-7ccd-4c28-a565-25d4b4a7dd73","Handler":null,"Name":"","Endpoint":""}
Jul 17 17:57:07.867799 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.867779    2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 17:57:07.867923 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.867820    2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 17:57:07.937577 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.937538    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:07.937767 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:07.937588    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:07.937767 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:07.937679    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:07.937881 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:07.937801    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:08.192072 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:08.191995    2555 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 17:57:08.937323 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:08.937289    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:08.937598 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:08.937438    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:09.196362 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:09.196269    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2" event={"ID":"1427b65f-7603-4dd8-9e30-d26adac520d5","Type":"ContainerStarted","Data":"e709787a505d4a2d1e05846cedba46c8250589cff795d085f6cbf1c16c626946"}
Jul 17 17:57:09.200261 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:09.200234    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"25858a5f123575d9219f5bb4735d7222843b37b6578141e3611f93fd2de22597"}
Jul 17 17:57:09.214158 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:09.214103    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6wtf2" podStartSLOduration=4.480516491 podStartE2EDuration="26.214084737s" totalImagesPullingTime="21.733568246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:09.213800209 +0000 UTC m=+26.799276462" watchObservedRunningTime="2026-07-17 17:57:09.214084737 +0000 UTC m=+26.799560990"
Jul 17 17:57:09.938237 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:09.938004    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:09.938440 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:09.938045    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:09.938440 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:09.938355    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:09.938440 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:09.938403    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:10.277845 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:10.277806    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:57:10.278304 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:10.277935    2555 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 17:57:10.278451 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:10.278433    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wbb86"
Jul 17 17:57:10.938050 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:10.937956    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:10.938216 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:10.938105    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:11.938093 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:11.937908    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:11.938734 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:11.937908    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:11.938734 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:11.938185    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:11.938734 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:11.938231    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:12.208231 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:12.208140    2555 generic.go:356] "Generic (PLEG): container finished" podID="8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1" containerID="93ea952084df6cc0cfdb578cfdcae4cde34bf179991d6c60deea5998d3bf4e38" exitCode=0
Jul 17 17:57:12.208402 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:12.208230    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerDied","Data":"93ea952084df6cc0cfdb578cfdcae4cde34bf179991d6c60deea5998d3bf4e38"}
Jul 17 17:57:12.212217 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:12.212190    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" event={"ID":"04f7b734-c54b-457d-ba62-ec6cf5ef10af","Type":"ContainerStarted","Data":"218483b4a209aa5535e5160f84d7f5b393af96b26f2426c48870028574c3680b"}
Jul 17 17:57:12.212464 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:12.212451    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:57:12.228147 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:12.228121    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:57:12.271653 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:12.271580    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6" podStartSLOduration=10.050099826 podStartE2EDuration="29.271563894s" totalImagesPullingTime="19.221464068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:12.270375178 +0000 UTC m=+29.855851430" watchObservedRunningTime="2026-07-17 17:57:12.271563894 +0000 UTC m=+29.857040167"
Jul 17 17:57:12.940257 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:12.940228    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:12.940744 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:12.940336    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:13.214983 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:13.214900    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:57:13.215119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:13.215026    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:57:13.229923 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:13.229894    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:57:13.937712 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:13.937675    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:13.937712 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:13.937713    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:13.937931 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:13.937788    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:13.937931 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:13.937888    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:14.218444 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:14.218355    2555 generic.go:356] "Generic (PLEG): container finished" podID="8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1" containerID="90be4b7d1b7275dac9b0dd841658962bb1ac56d0d285ac7862e30b68b02fd3a8" exitCode=0
Jul 17 17:57:14.218852 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:14.218454    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerDied","Data":"90be4b7d1b7275dac9b0dd841658962bb1ac56d0d285ac7862e30b68b02fd3a8"}
Jul 17 17:57:14.937595 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:14.937565    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:14.937802 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:14.937693    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:15.937791 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:15.937748    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:15.938197 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:15.937852    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:15.938197 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:15.937904    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:15.938197 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:15.937992    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:16.224604 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:16.224512    2555 generic.go:356] "Generic (PLEG): container finished" podID="8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1" containerID="3f20dc551fe87dafa082fe934d58fa6e482cb070ee633851e741aa4599029450" exitCode=0
Jul 17 17:57:16.224604 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:16.224557    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerDied","Data":"3f20dc551fe87dafa082fe934d58fa6e482cb070ee633851e741aa4599029450"}
Jul 17 17:57:16.566448 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:16.566411    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:16.566675 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:16.566457    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:16.566675 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.566577    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 17:57:16.566675 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.566607    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 17:57:16.566675 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.566620    2555 projected.go:196] Error preparing data for projected volume kube-api-access-6w2nd for pod openshift-network-diagnostics/network-check-target-c9xd4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:57:16.566675 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.566584    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 17:57:16.566876 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.566689    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd podName:1031a638-0acf-4023-91f8-ba906ca32530 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:48.566670825 +0000 UTC m=+66.152147057 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6w2nd" (UniqueName: "kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd") pod "network-check-target-c9xd4" (UID: "1031a638-0acf-4023-91f8-ba906ca32530") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 17:57:16.566876 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.566721    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret podName:0c868384-1df1-438e-8e7c-6a7d578a58af nodeName:}" failed. No retries permitted until 2026-07-17 17:57:48.566703995 +0000 UTC m=+66.152180237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret") pod "global-pull-secret-syncer-snwmp" (UID: "0c868384-1df1-438e-8e7c-6a7d578a58af") : object "kube-system"/"original-pull-secret" not registered
Jul 17 17:57:16.666965 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:16.666928    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:16.667139 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.667075    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:57:16.667182 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.667140    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs podName:b6b5f8c0-20a0-4d96-9ab4-b0e256242982 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:48.667121934 +0000 UTC m=+66.252598168 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs") pod "network-metrics-daemon-5n67b" (UID: "b6b5f8c0-20a0-4d96-9ab4-b0e256242982") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 17:57:16.937676 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:16.937581    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:16.937822 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:16.937711    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:17.937654 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:17.937609    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:17.937844 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:17.937609    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:17.937844 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:17.937757    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:17.937844 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:17.937819    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:18.938015 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:18.937978    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:18.938458 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:18.938108    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:19.937810 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:19.937771    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:19.937999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:19.937771    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:19.937999 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:19.937895    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:19.937999 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:19.937947    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:20.938072 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:20.938034    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:20.938422 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:20.938155    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:21.240053 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:21.240019    2555 generic.go:356] "Generic (PLEG): container finished" podID="8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1" containerID="c70e191a4807aa6012e06cb6867d15fe28b2893fa6d2c98195aed5b18fab9adf" exitCode=0
Jul 17 17:57:21.240218 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:21.240070    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerDied","Data":"c70e191a4807aa6012e06cb6867d15fe28b2893fa6d2c98195aed5b18fab9adf"}
Jul 17 17:57:21.937353 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:21.937317    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:21.937353 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:21.937355    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:21.937563 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:21.937423    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:21.937563 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:21.937489    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:22.247104 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:22.247069    2555 generic.go:356] "Generic (PLEG): container finished" podID="8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1" containerID="7af324427dac4017712dffb1a15c4cae92ae22e982dbc5f6e6d874be5224c6a9" exitCode=0
Jul 17 17:57:22.247495 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:22.247118    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerDied","Data":"7af324427dac4017712dffb1a15c4cae92ae22e982dbc5f6e6d874be5224c6a9"}
Jul 17 17:57:22.939428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:22.939186    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:22.939603 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:22.939528    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:23.252795 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:23.252761    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" event={"ID":"8e9a49e5-e8c3-4fcf-8517-a6024e41f7a1","Type":"ContainerStarted","Data":"d4bdd6d92471781e98c309dad615dff9e49804ccd7ab7d7a9ba09873f911b8f7"}
Jul 17 17:57:23.274803 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:23.274750    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v8q9v" podStartSLOduration=8.995410135 podStartE2EDuration="40.274732856s" totalImagesPullingTime="31.279322721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:23.273303481 +0000 UTC m=+40.858779733" watchObservedRunningTime="2026-07-17 17:57:23.274732856 +0000 UTC m=+40.860209107"
Jul 17 17:57:23.937360 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:23.937321    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:23.937538 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:23.937321    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:23.937538 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:23.937429    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:23.937538 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:23.937494    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:24.937566 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:24.937532    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:24.938025 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:24.937685    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:25.937017 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:25.936980    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:25.937225 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:25.936980    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:25.937225 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:25.937083    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:25.937225 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:25.937155    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:26.937624 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:26.937585    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:26.938024 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:26.937729    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:27.937635 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:27.937593    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:27.937806 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:27.937618    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:27.937806 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:27.937772    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:27.938279 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:27.937877    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:28.271228 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:28.271198    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c9xd4"]
Jul 17 17:57:28.271443 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:28.271309    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:28.271443 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:28.271416    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:28.274397 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:28.274358    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-snwmp"]
Jul 17 17:57:28.274518 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:28.274468    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:28.274598 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:28.274569    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:28.275169 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:28.275144    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5n67b"]
Jul 17 17:57:28.275292 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:28.275247    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:28.275508 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:28.275356    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:29.937444 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:29.937400    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:29.938131 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:29.937409    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:29.938131 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:29.937524    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:29.938131 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:29.937537    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:29.938131 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:29.937616    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:29.938131 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:29.937755    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:31.937864 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:31.937829    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:31.937864 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:31.937869    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:31.938307 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:31.937870    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:31.938307 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:31.937949    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5n67b" podUID="b6b5f8c0-20a0-4d96-9ab4-b0e256242982"
Jul 17 17:57:31.938307 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:31.938039    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snwmp" podUID="0c868384-1df1-438e-8e7c-6a7d578a58af"
Jul 17 17:57:31.938307 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:31.938097    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c9xd4" podUID="1031a638-0acf-4023-91f8-ba906ca32530"
Jul 17 17:57:33.153950 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.153722    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeReady"
Jul 17 17:57:33.154368 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.154000    2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 17:57:33.196525 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.196489    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nmplq"]
Jul 17 17:57:33.228277 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.228247    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4hlgg"]
Jul 17 17:57:33.228428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.228403    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.230857 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.230834    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 17:57:33.231015 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.230872    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s6gdc\""
Jul 17 17:57:33.231015 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.230834    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 17:57:33.240881 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.240857    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nmplq"]
Jul 17 17:57:33.241008 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.240897    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4hlgg"]
Jul 17 17:57:33.241008 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.240918    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r26rr"]
Jul 17 17:57:33.241008 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.240967    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.243178 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.243156    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 17:57:33.243407 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.243392    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hnrcx\""
Jul 17 17:57:33.243522 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.243501    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 17:57:33.243590 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.243573    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 17:57:33.261137 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.261102    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r26rr"]
Jul 17 17:57:33.261323 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.261237    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.263680 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.263630    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 17:57:33.263827 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.263694    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 17:57:33.263827 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.263727    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 17:57:33.263827 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.263697    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ns7ff\""
Jul 17 17:57:33.264003 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.263875    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 17:57:33.282572 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282545    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/725734b7-d548-4d5f-b25f-9fa3e81bddde-cert\") pod \"ingress-canary-4hlgg\" (UID: \"725734b7-d548-4d5f-b25f-9fa3e81bddde\") " pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.282743 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282591    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxqh\" (UniqueName: \"kubernetes.io/projected/b1df721d-260e-4fa4-806b-746e343409f0-kube-api-access-jgxqh\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.282743 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282623    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb5s5\" (UniqueName: \"kubernetes.io/projected/26ac9501-351d-48de-8572-b583ac2d9a70-kube-api-access-zb5s5\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.282743 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282691    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26ac9501-351d-48de-8572-b583ac2d9a70-config-volume\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.282743 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282730    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ac9501-351d-48de-8572-b583ac2d9a70-tmp-dir\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.282868 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282759    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1df721d-260e-4fa4-806b-746e343409f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.282868 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282785    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26ac9501-351d-48de-8572-b583ac2d9a70-metrics-tls\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.282868 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282805    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1df721d-260e-4fa4-806b-746e343409f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.282868 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282858    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlkns\" (UniqueName: \"kubernetes.io/projected/725734b7-d548-4d5f-b25f-9fa3e81bddde-kube-api-access-hlkns\") pod \"ingress-canary-4hlgg\" (UID: \"725734b7-d548-4d5f-b25f-9fa3e81bddde\") " pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.282984 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282876    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1df721d-260e-4fa4-806b-746e343409f0-data-volume\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.282984 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.282906    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b1df721d-260e-4fa4-806b-746e343409f0-crio-socket\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.383999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.383966    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26ac9501-351d-48de-8572-b583ac2d9a70-config-volume\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.383999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.383997    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ac9501-351d-48de-8572-b583ac2d9a70-tmp-dir\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.384244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384014    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1df721d-260e-4fa4-806b-746e343409f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.384244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384037    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26ac9501-351d-48de-8572-b583ac2d9a70-metrics-tls\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.384244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384059    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1df721d-260e-4fa4-806b-746e343409f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.384244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384094    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlkns\" (UniqueName: \"kubernetes.io/projected/725734b7-d548-4d5f-b25f-9fa3e81bddde-kube-api-access-hlkns\") pod \"ingress-canary-4hlgg\" (UID: \"725734b7-d548-4d5f-b25f-9fa3e81bddde\") " pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.384244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384117    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1df721d-260e-4fa4-806b-746e343409f0-data-volume\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.384244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384168    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b1df721d-260e-4fa4-806b-746e343409f0-crio-socket\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.384244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384209    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/725734b7-d548-4d5f-b25f-9fa3e81bddde-cert\") pod \"ingress-canary-4hlgg\" (UID: \"725734b7-d548-4d5f-b25f-9fa3e81bddde\") " pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.384574 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384253    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxqh\" (UniqueName: \"kubernetes.io/projected/b1df721d-260e-4fa4-806b-746e343409f0-kube-api-access-jgxqh\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.384574 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384281    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb5s5\" (UniqueName: \"kubernetes.io/projected/26ac9501-351d-48de-8572-b583ac2d9a70-kube-api-access-zb5s5\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.384574 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384439    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26ac9501-351d-48de-8572-b583ac2d9a70-tmp-dir\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.384765 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384588    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b1df721d-260e-4fa4-806b-746e343409f0-crio-socket\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.384821 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384763    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/26ac9501-351d-48de-8572-b583ac2d9a70-config-volume\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.384873 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384857    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1df721d-260e-4fa4-806b-746e343409f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.384946 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.384929    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1df721d-260e-4fa4-806b-746e343409f0-data-volume\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.388403 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.388378    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/26ac9501-351d-48de-8572-b583ac2d9a70-metrics-tls\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.388403 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.388402    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/725734b7-d548-4d5f-b25f-9fa3e81bddde-cert\") pod \"ingress-canary-4hlgg\" (UID: \"725734b7-d548-4d5f-b25f-9fa3e81bddde\") " pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.388548 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.388407    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1df721d-260e-4fa4-806b-746e343409f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.391938 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.391916    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxqh\" (UniqueName: \"kubernetes.io/projected/b1df721d-260e-4fa4-806b-746e343409f0-kube-api-access-jgxqh\") pod \"insights-runtime-extractor-r26rr\" (UID: \"b1df721d-260e-4fa4-806b-746e343409f0\") " pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.392085 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.392002    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb5s5\" (UniqueName: \"kubernetes.io/projected/26ac9501-351d-48de-8572-b583ac2d9a70-kube-api-access-zb5s5\") pod \"dns-default-nmplq\" (UID: \"26ac9501-351d-48de-8572-b583ac2d9a70\") " pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.392182 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.392165    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlkns\" (UniqueName: \"kubernetes.io/projected/725734b7-d548-4d5f-b25f-9fa3e81bddde-kube-api-access-hlkns\") pod \"ingress-canary-4hlgg\" (UID: \"725734b7-d548-4d5f-b25f-9fa3e81bddde\") " pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.538882 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.538850    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:33.548987 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.548956    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4hlgg"
Jul 17 17:57:33.570155 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.570132    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r26rr"
Jul 17 17:57:33.744966 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.744937    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nmplq"]
Jul 17 17:57:33.748174 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.748149    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4hlgg"]
Jul 17 17:57:33.750135 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:33.750110    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ac9501_351d_48de_8572_b583ac2d9a70.slice/crio-fe52a8b121e07dd0b10f1798647838a4af8e154968a943465ba435de385edefd WatchSource:0}: Error finding container fe52a8b121e07dd0b10f1798647838a4af8e154968a943465ba435de385edefd: Status 404 returned error can't find the container with id fe52a8b121e07dd0b10f1798647838a4af8e154968a943465ba435de385edefd
Jul 17 17:57:33.751851 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:33.751831    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725734b7_d548_4d5f_b25f_9fa3e81bddde.slice/crio-ee33b2b3ef3866c330ee987861cce53179874a1af260e33e987b9a292ea0acd0 WatchSource:0}: Error finding container ee33b2b3ef3866c330ee987861cce53179874a1af260e33e987b9a292ea0acd0: Status 404 returned error can't find the container with id ee33b2b3ef3866c330ee987861cce53179874a1af260e33e987b9a292ea0acd0
Jul 17 17:57:33.759911 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.759888    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r26rr"]
Jul 17 17:57:33.776234 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:33.776208    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1df721d_260e_4fa4_806b_746e343409f0.slice/crio-892593b82c0c226bf306c704696c218284719088c88dba783d6007d7829d0706 WatchSource:0}: Error finding container 892593b82c0c226bf306c704696c218284719088c88dba783d6007d7829d0706: Status 404 returned error can't find the container with id 892593b82c0c226bf306c704696c218284719088c88dba783d6007d7829d0706
Jul 17 17:57:33.937526 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.937439    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:33.937526 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.937469    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:33.937526 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.937488    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:33.940257 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.940236    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 17:57:33.941030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.941008    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 17:57:33.941135 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.941032    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 17:57:33.941135 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.941009    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jb7qg\""
Jul 17 17:57:33.941135 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.941012    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dn68c\""
Jul 17 17:57:33.941135 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:33.941085    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 17:57:34.280469 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:34.280432    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r26rr" event={"ID":"b1df721d-260e-4fa4-806b-746e343409f0","Type":"ContainerStarted","Data":"892593b82c0c226bf306c704696c218284719088c88dba783d6007d7829d0706"}
Jul 17 17:57:34.282097 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:34.282018    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nmplq" event={"ID":"26ac9501-351d-48de-8572-b583ac2d9a70","Type":"ContainerStarted","Data":"fe52a8b121e07dd0b10f1798647838a4af8e154968a943465ba435de385edefd"}
Jul 17 17:57:34.283380 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:34.283356    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4hlgg" event={"ID":"725734b7-d548-4d5f-b25f-9fa3e81bddde","Type":"ContainerStarted","Data":"ee33b2b3ef3866c330ee987861cce53179874a1af260e33e987b9a292ea0acd0"}
Jul 17 17:57:37.459388 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.459187    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"]
Jul 17 17:57:37.538400 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.538369    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"]
Jul 17 17:57:37.538575 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.538505    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.541471 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.541366    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 17 17:57:37.541471 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.541404    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 17:57:37.541471 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.541441    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-sd986\""
Jul 17 17:57:37.542367 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.542346    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 17:57:37.542471 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.542410    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 17 17:57:37.542699 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.542682    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 17:57:37.622403 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.622358    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.622569 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.622461    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.622569 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.622538    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlsxf\" (UniqueName: \"kubernetes.io/projected/8c42270d-80c5-4d97-b51a-7dacb92b87c0-kube-api-access-nlsxf\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.622569 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.622562    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c42270d-80c5-4d97-b51a-7dacb92b87c0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.723105 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.723030    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.723105 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.723093    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlsxf\" (UniqueName: \"kubernetes.io/projected/8c42270d-80c5-4d97-b51a-7dacb92b87c0-kube-api-access-nlsxf\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.723326 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.723117    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c42270d-80c5-4d97-b51a-7dacb92b87c0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.723326 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.723152    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.723326 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:37.723200    2555 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 17 17:57:37.723326 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:37.723275    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls podName:8c42270d-80c5-4d97-b51a-7dacb92b87c0 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:38.223254312 +0000 UTC m=+55.808730543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-stbxg" (UID: "8c42270d-80c5-4d97-b51a-7dacb92b87c0") : secret "prometheus-operator-tls" not found
Jul 17 17:57:37.723918 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.723896    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c42270d-80c5-4d97-b51a-7dacb92b87c0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.726887 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.726866    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:37.732089 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:37.732061    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlsxf\" (UniqueName: \"kubernetes.io/projected/8c42270d-80c5-4d97-b51a-7dacb92b87c0-kube-api-access-nlsxf\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:38.227445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:38.227407    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:38.227618 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:38.227576    2555 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 17 17:57:38.227695 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:38.227681    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls podName:8c42270d-80c5-4d97-b51a-7dacb92b87c0 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:39.227633896 +0000 UTC m=+56.813110140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-stbxg" (UID: "8c42270d-80c5-4d97-b51a-7dacb92b87c0") : secret "prometheus-operator-tls" not found
Jul 17 17:57:39.237495 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:39.237458    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:39.240400 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:39.240370    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c42270d-80c5-4d97-b51a-7dacb92b87c0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-stbxg\" (UID: \"8c42270d-80c5-4d97-b51a-7dacb92b87c0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:39.350501 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:39.350467    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"
Jul 17 17:57:39.565612 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:39.565531    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-stbxg"]
Jul 17 17:57:39.571438 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:39.571406    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c42270d_80c5_4d97_b51a_7dacb92b87c0.slice/crio-3d4fa9f59f458433cb9b244f5b57569c1494788e3d1978c6c2c04704671d931a WatchSource:0}: Error finding container 3d4fa9f59f458433cb9b244f5b57569c1494788e3d1978c6c2c04704671d931a: Status 404 returned error can't find the container with id 3d4fa9f59f458433cb9b244f5b57569c1494788e3d1978c6c2c04704671d931a
Jul 17 17:57:40.302981 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.302943    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nmplq" event={"ID":"26ac9501-351d-48de-8572-b583ac2d9a70","Type":"ContainerStarted","Data":"ab3e2432ee14d7828b3fecf0838463ecacac3f6931f8d606ca5dd9274fe25582"}
Jul 17 17:57:40.303457 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.303043    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nmplq" event={"ID":"26ac9501-351d-48de-8572-b583ac2d9a70","Type":"ContainerStarted","Data":"ecd68575d8ae1f1c6eee12927a4d9707db2223afcfe75f70eeb7b7ed37c18283"}
Jul 17 17:57:40.303457 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.303217    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:40.305660 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.305578    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4hlgg" event={"ID":"725734b7-d548-4d5f-b25f-9fa3e81bddde","Type":"ContainerStarted","Data":"d6c356584efd514dccd276cfde1da4f5489073fc67eb407298149c800d938a63"}
Jul 17 17:57:40.307613 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.307556    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg" event={"ID":"8c42270d-80c5-4d97-b51a-7dacb92b87c0","Type":"ContainerStarted","Data":"3d4fa9f59f458433cb9b244f5b57569c1494788e3d1978c6c2c04704671d931a"}
Jul 17 17:57:40.309554 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.309495    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r26rr" event={"ID":"b1df721d-260e-4fa4-806b-746e343409f0","Type":"ContainerStarted","Data":"bf8c7c73a72b4861368937a8461916668931a9bf47af25b16a6190b9b454582d"}
Jul 17 17:57:40.322221 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.322062    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nmplq" podStartSLOduration=1.652979379 podStartE2EDuration="7.322042255s" totalImagesPullingTime="5.669062876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:40.320972402 +0000 UTC m=+57.906448658" watchObservedRunningTime="2026-07-17 17:57:40.322042255 +0000 UTC m=+57.907518509"
Jul 17 17:57:40.337664 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:40.337165    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4hlgg" podStartSLOduration=1.669844616 podStartE2EDuration="7.337143653s" totalImagesPullingTime="5.667299037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:40.33711535 +0000 UTC m=+57.922591627" watchObservedRunningTime="2026-07-17 17:57:40.337143653 +0000 UTC m=+57.922619906"
Jul 17 17:57:41.315708 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:41.315674    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg" event={"ID":"8c42270d-80c5-4d97-b51a-7dacb92b87c0","Type":"ContainerStarted","Data":"0fbe8419438753c02240ce3b2b4423886f25891eb25dc701050410de4fc46f1a"}
Jul 17 17:57:41.316166 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:41.315738    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg" event={"ID":"8c42270d-80c5-4d97-b51a-7dacb92b87c0","Type":"ContainerStarted","Data":"9f77c62d30c2e02a78b20d98cfc269bbd22e884d697df843e252a940240ad302"}
Jul 17 17:57:41.331960 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:41.331904    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-stbxg" podStartSLOduration=2.974780331 podStartE2EDuration="4.33188141s" totalImagesPullingTime="1.357101079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:41.330681591 +0000 UTC m=+58.916157844" watchObservedRunningTime="2026-07-17 17:57:41.33188141 +0000 UTC m=+58.917357664"
Jul 17 17:57:42.786914 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.786878    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"]
Jul 17 17:57:42.796786 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.796756    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.799318 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.799289    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 17 17:57:42.799489 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.799373    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 17 17:57:42.799489 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.799418    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2gnx4\""
Jul 17 17:57:42.802810 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.802737    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"]
Jul 17 17:57:42.814948 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.814921    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gpb99"]
Jul 17 17:57:42.817876 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.817852    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.820434 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.820407    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 17 17:57:42.820780 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.820762    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qrpkp\""
Jul 17 17:57:42.820979 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.820961    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 17 17:57:42.821212 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.821160    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 17 17:57:42.823460 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.823435    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ldk6v"]
Jul 17 17:57:42.826439 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.826366    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.828537 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.828519    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 17:57:42.829718 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.829700    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 17:57:42.830013 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.829998    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4h8ql\""
Jul 17 17:57:42.830276 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.830253    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 17:57:42.831153 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.831133    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gpb99"]
Jul 17 17:57:42.865683 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865618    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.865835 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865709    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.865835 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865734    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.865835 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865769    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.865835 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865792    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8svw\" (UniqueName: \"kubernetes.io/projected/41157244-5641-4575-a240-93418d028431-kube-api-access-d8svw\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.865835 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865817    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41157244-5641-4575-a240-93418d028431-node-exporter-textfile\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.866084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865840    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hk9c\" (UniqueName: \"kubernetes.io/projected/27b36904-0e33-4a42-8ea4-9b8ae5d58955-kube-api-access-4hk9c\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.866084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865895    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.866084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.865966    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27b36904-0e33-4a42-8ea4-9b8ae5d58955-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.866084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866022    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-node-exporter-wtmp\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.866084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866044    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.866084 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866068    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjjhf\" (UniqueName: \"kubernetes.io/projected/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-api-access-gjjhf\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.866329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866103    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-sys\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.866329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866132    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.866329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866190    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-root\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.866329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866225    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41157244-5641-4575-a240-93418d028431-node-exporter-accelerators-collector-config\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.866329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866254    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.866329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866309    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41157244-5641-4575-a240-93418d028431-metrics-client-ca\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.866547 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.866347    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-tls\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.967122 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967085    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.967122 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967128    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27b36904-0e33-4a42-8ea4-9b8ae5d58955-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.967383 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967182    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-node-exporter-wtmp\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.967383 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967206    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.967383 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967230    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjjhf\" (UniqueName: \"kubernetes.io/projected/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-api-access-gjjhf\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.967383 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:42.967240    2555 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 17 17:57:42.967383 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:42.967303    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-tls podName:d4f4292b-cc90-4769-bfe6-c49b3b7917c0 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:43.467283787 +0000 UTC m=+61.052760036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-gpb99" (UID: "d4f4292b-cc90-4769-bfe6-c49b3b7917c0") : secret "kube-state-metrics-tls" not found
Jul 17 17:57:42.967794 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967384    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-node-exporter-wtmp\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.967794 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967461    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-sys\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.967794 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967496    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.967794 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967540    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-root\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.967794 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967592    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-sys\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.967794 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967604    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41157244-5641-4575-a240-93418d028431-root\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.968041 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.967635    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41157244-5641-4575-a240-93418d028431-node-exporter-accelerators-collector-config\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.968092 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968036    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.968141 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968088    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41157244-5641-4575-a240-93418d028431-metrics-client-ca\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.968141 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968131    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-tls\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.968238 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968167    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.968238 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968214    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.968329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968245    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.968329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968292    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.968329 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968317    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8svw\" (UniqueName: \"kubernetes.io/projected/41157244-5641-4575-a240-93418d028431-kube-api-access-d8svw\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.968462 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968352    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41157244-5641-4575-a240-93418d028431-node-exporter-textfile\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.968462 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968353    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27b36904-0e33-4a42-8ea4-9b8ae5d58955-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.968462 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968384    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hk9c\" (UniqueName: \"kubernetes.io/projected/27b36904-0e33-4a42-8ea4-9b8ae5d58955-kube-api-access-4hk9c\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.968595 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968526    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.968595 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:42.968576    2555 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 17 17:57:42.968595 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.968585    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.968752 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:42.968666    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-tls podName:27b36904-0e33-4a42-8ea4-9b8ae5d58955 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:43.468618535 +0000 UTC m=+61.054094766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-gw5f5" (UID: "27b36904-0e33-4a42-8ea4-9b8ae5d58955") : secret "openshift-state-metrics-tls" not found
Jul 17 17:57:42.968807 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:42.968763    2555 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 17:57:42.968853 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:42.968811    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-tls podName:41157244-5641-4575-a240-93418d028431 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:43.468797648 +0000 UTC m=+61.054273883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-tls") pod "node-exporter-ldk6v" (UID: "41157244-5641-4575-a240-93418d028431") : secret "node-exporter-tls" not found
Jul 17 17:57:42.969211 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.969169    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41157244-5641-4575-a240-93418d028431-node-exporter-accelerators-collector-config\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.969791 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.969461    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41157244-5641-4575-a240-93418d028431-node-exporter-textfile\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.969791 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.969711    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.969791 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.969749    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41157244-5641-4575-a240-93418d028431-metrics-client-ca\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.973678 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.972190    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:42.973678 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.972248    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.973678 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.972251    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.977017 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.976986    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjjhf\" (UniqueName: \"kubernetes.io/projected/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-api-access-gjjhf\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:42.977101 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.976986    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hk9c\" (UniqueName: \"kubernetes.io/projected/27b36904-0e33-4a42-8ea4-9b8ae5d58955-kube-api-access-4hk9c\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:42.977308 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:42.977285    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8svw\" (UniqueName: \"kubernetes.io/projected/41157244-5641-4575-a240-93418d028431-kube-api-access-d8svw\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:43.323974 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.323888    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r26rr" event={"ID":"b1df721d-260e-4fa4-806b-746e343409f0","Type":"ContainerStarted","Data":"9fef8e0cef66d9e7cc490113fe62bd25e4c08caaccbd9f0a6d7ebbd37dae8cce"}
Jul 17 17:57:43.471717 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.471683    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:43.471894 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.471783    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-tls\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:43.471894 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.471824    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:43.474807 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.474776    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41157244-5641-4575-a240-93418d028431-node-exporter-tls\") pod \"node-exporter-ldk6v\" (UID: \"41157244-5641-4575-a240-93418d028431\") " pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:43.474934 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.474818    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4f4292b-cc90-4769-bfe6-c49b3b7917c0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-gpb99\" (UID: \"d4f4292b-cc90-4769-bfe6-c49b3b7917c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:43.475421 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.475401    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/27b36904-0e33-4a42-8ea4-9b8ae5d58955-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-gw5f5\" (UID: \"27b36904-0e33-4a42-8ea4-9b8ae5d58955\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:43.708806 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.708723    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"
Jul 17 17:57:43.729572 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.729525    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99"
Jul 17 17:57:43.738298 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.738268    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ldk6v"
Jul 17 17:57:43.751813 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:43.751772    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41157244_5641_4575_a240_93418d028431.slice/crio-bdbfd04023e86862232cdce316107121a7f49c07ed6b8851912bfb9153ba1d26 WatchSource:0}: Error finding container bdbfd04023e86862232cdce316107121a7f49c07ed6b8851912bfb9153ba1d26: Status 404 returned error can't find the container with id bdbfd04023e86862232cdce316107121a7f49c07ed6b8851912bfb9153ba1d26
Jul 17 17:57:43.879501 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.879414    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5"]
Jul 17 17:57:43.883636 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:43.883571    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b36904_0e33_4a42_8ea4_9b8ae5d58955.slice/crio-e8672ae47e3f8b998504b935f22e83d21ec93db5686e679c53a846ef0d6224d0 WatchSource:0}: Error finding container e8672ae47e3f8b998504b935f22e83d21ec93db5686e679c53a846ef0d6224d0: Status 404 returned error can't find the container with id e8672ae47e3f8b998504b935f22e83d21ec93db5686e679c53a846ef0d6224d0
Jul 17 17:57:43.896238 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:43.896216    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-gpb99"]
Jul 17 17:57:43.900791 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:43.900762    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f4292b_cc90_4769_bfe6_c49b3b7917c0.slice/crio-27928ecefdf71d52d64e0e7ecf359bce12d71f8db3262ee29ce9cc3236ad18ae WatchSource:0}: Error finding container 27928ecefdf71d52d64e0e7ecf359bce12d71f8db3262ee29ce9cc3236ad18ae: Status 404 returned error can't find the container with id 27928ecefdf71d52d64e0e7ecf359bce12d71f8db3262ee29ce9cc3236ad18ae
Jul 17 17:57:44.331788 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:44.331753    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5" event={"ID":"27b36904-0e33-4a42-8ea4-9b8ae5d58955","Type":"ContainerStarted","Data":"ee53389b12ff385e337d82edded4fcdcbc4a36d9c63be826d48ce712737fd42b"}
Jul 17 17:57:44.331993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:44.331829    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5" event={"ID":"27b36904-0e33-4a42-8ea4-9b8ae5d58955","Type":"ContainerStarted","Data":"4c40b62524eb49129fa675aaa0f0b63807aa5287f33947b340989d82d61d0dd7"}
Jul 17 17:57:44.331993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:44.331865    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5" event={"ID":"27b36904-0e33-4a42-8ea4-9b8ae5d58955","Type":"ContainerStarted","Data":"e8672ae47e3f8b998504b935f22e83d21ec93db5686e679c53a846ef0d6224d0"}
Jul 17 17:57:44.333679 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:44.333630    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99" event={"ID":"d4f4292b-cc90-4769-bfe6-c49b3b7917c0","Type":"ContainerStarted","Data":"27928ecefdf71d52d64e0e7ecf359bce12d71f8db3262ee29ce9cc3236ad18ae"}
Jul 17 17:57:44.334953 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:44.334927    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ldk6v" event={"ID":"41157244-5641-4575-a240-93418d028431","Type":"ContainerStarted","Data":"bdbfd04023e86862232cdce316107121a7f49c07ed6b8851912bfb9153ba1d26"}
Jul 17 17:57:45.244058 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.244029    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm5c6"
Jul 17 17:57:45.780454 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.780391    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"]
Jul 17 17:57:45.790095 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.790058    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.792741 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.792709    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5lq928qkg9nu\""
Jul 17 17:57:45.792741 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.792738    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 17 17:57:45.792999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.792747    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 17 17:57:45.792999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.792788    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2gvkd\""
Jul 17 17:57:45.792999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.792709    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 17 17:57:45.792999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.792903    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 17 17:57:45.792999 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.792980    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 17 17:57:45.796663 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.796621    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"]
Jul 17 17:57:45.895574 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895544    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-metrics-client-ca\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.895760 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895582    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.895760 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895620    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.895760 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895668    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fgzf\" (UniqueName: \"kubernetes.io/projected/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-kube-api-access-5fgzf\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.895908 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895754    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-grpc-tls\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.895908 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895813    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-tls\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.895908 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895836    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.895908 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.895887    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996247 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996208    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996247 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996251    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fgzf\" (UniqueName: \"kubernetes.io/projected/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-kube-api-access-5fgzf\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996279    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-grpc-tls\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996307    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-tls\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996323    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996348    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996401    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-metrics-client-ca\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.996491 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.996429    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.997295 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.997227    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-metrics-client-ca\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.999550 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.999519    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.999674 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.999562    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-tls\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.999863 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.999833    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.999946 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.999871    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:45.999946 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:45.999923    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:46.000080 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:46.000040    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-secret-grpc-tls\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:46.004515 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:46.004489    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fgzf\" (UniqueName: \"kubernetes.io/projected/8dd1f855-6db3-4d9f-935e-9dfa0e925e65-kube-api-access-5fgzf\") pod \"thanos-querier-6cfb69d5b5-cfl9f\" (UID: \"8dd1f855-6db3-4d9f-935e-9dfa0e925e65\") " pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:46.101424 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:46.101326    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:47.568137 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.568111    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"]
Jul 17 17:57:47.585497 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.585441    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"]
Jul 17 17:57:47.588459 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.588387    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:47.591474 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.591374    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 17:57:47.591474 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.591451    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pdf2z\""
Jul 17 17:57:47.606824 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.606793    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"]
Jul 17 17:57:47.712198 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.712164    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b02a87df-bb7d-4ca4-b7e2-0babbac79342-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-z9w7t\" (UID: \"b02a87df-bb7d-4ca4-b7e2-0babbac79342\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:47.712198 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.712204    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-297fl\" (UniqueName: \"kubernetes.io/projected/b02a87df-bb7d-4ca4-b7e2-0babbac79342-kube-api-access-297fl\") pod \"monitoring-plugin-7b7b7bc8bf-z9w7t\" (UID: \"b02a87df-bb7d-4ca4-b7e2-0babbac79342\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:47.813530 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.813491    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b02a87df-bb7d-4ca4-b7e2-0babbac79342-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-z9w7t\" (UID: \"b02a87df-bb7d-4ca4-b7e2-0babbac79342\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:47.813749 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.813536    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-297fl\" (UniqueName: \"kubernetes.io/projected/b02a87df-bb7d-4ca4-b7e2-0babbac79342-kube-api-access-297fl\") pod \"monitoring-plugin-7b7b7bc8bf-z9w7t\" (UID: \"b02a87df-bb7d-4ca4-b7e2-0babbac79342\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:47.813749 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:47.813675    2555 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 17 17:57:47.813881 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:57:47.813754    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b02a87df-bb7d-4ca4-b7e2-0babbac79342-monitoring-plugin-cert podName:b02a87df-bb7d-4ca4-b7e2-0babbac79342 nodeName:}" failed. No retries permitted until 2026-07-17 17:57:48.313731531 +0000 UTC m=+65.899207764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b02a87df-bb7d-4ca4-b7e2-0babbac79342-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-z9w7t" (UID: "b02a87df-bb7d-4ca4-b7e2-0babbac79342") : secret "monitoring-plugin-cert" not found
Jul 17 17:57:47.823509 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:47.823480    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-297fl\" (UniqueName: \"kubernetes.io/projected/b02a87df-bb7d-4ca4-b7e2-0babbac79342-kube-api-access-297fl\") pod \"monitoring-plugin-7b7b7bc8bf-z9w7t\" (UID: \"b02a87df-bb7d-4ca4-b7e2-0babbac79342\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:48.317475 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.317436    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b02a87df-bb7d-4ca4-b7e2-0babbac79342-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-z9w7t\" (UID: \"b02a87df-bb7d-4ca4-b7e2-0babbac79342\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:48.320043 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.320015    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b02a87df-bb7d-4ca4-b7e2-0babbac79342-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-z9w7t\" (UID: \"b02a87df-bb7d-4ca4-b7e2-0babbac79342\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:48.356167 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.356131    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5" event={"ID":"27b36904-0e33-4a42-8ea4-9b8ae5d58955","Type":"ContainerStarted","Data":"700031b4a07209e723f1ad8c0a55fd6a20ac7a9282d3049b4ee10c1755a7eb46"}
Jul 17 17:57:48.357346 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.357324    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" event={"ID":"8dd1f855-6db3-4d9f-935e-9dfa0e925e65","Type":"ContainerStarted","Data":"dc5ecc3a717e9633134a722c52e714157d5f64cb1ea42e23a0b0f1c62826c2b7"}
Jul 17 17:57:48.359132 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.359103    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99" event={"ID":"d4f4292b-cc90-4769-bfe6-c49b3b7917c0","Type":"ContainerStarted","Data":"2150b05772e4d3ceeb464fd19d4997c655020b110e9ffc3ad4a549783c45e3ac"}
Jul 17 17:57:48.359240 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.359155    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99" event={"ID":"d4f4292b-cc90-4769-bfe6-c49b3b7917c0","Type":"ContainerStarted","Data":"1eb71b101cdf2bd79e577e1b2b73b95b217a9af580d94975cf848c9bf6eb8902"}
Jul 17 17:57:48.359240 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.359175    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99" event={"ID":"d4f4292b-cc90-4769-bfe6-c49b3b7917c0","Type":"ContainerStarted","Data":"1f6f78ee67f264a7e28d3c7b215571bad0f900786e4009085d2e790dca9e91db"}
Jul 17 17:57:48.361015 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.360996    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r26rr" event={"ID":"b1df721d-260e-4fa4-806b-746e343409f0","Type":"ContainerStarted","Data":"f604c4561626e8fdccd707c2d59ef2ba6b2e32003b0f30d8f53412128316dff3"}
Jul 17 17:57:48.362430 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.362409    2555 generic.go:356] "Generic (PLEG): container finished" podID="41157244-5641-4575-a240-93418d028431" containerID="9f4aedbeb7c1f4ea2abcafbc3d586443b8d488c7a386ca705857648dd3269e21" exitCode=0
Jul 17 17:57:48.362521 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.362438    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ldk6v" event={"ID":"41157244-5641-4575-a240-93418d028431","Type":"ContainerDied","Data":"9f4aedbeb7c1f4ea2abcafbc3d586443b8d488c7a386ca705857648dd3269e21"}
Jul 17 17:57:48.375276 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.375230    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-gw5f5" podStartSLOduration=3.016973053 podStartE2EDuration="6.375214262s" totalImagesPullingTime="3.358241209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:48.374278034 +0000 UTC m=+65.959754306" watchObservedRunningTime="2026-07-17 17:57:48.375214262 +0000 UTC m=+65.960690517"
Jul 17 17:57:48.391803 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.391749    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-gpb99" podStartSLOduration=2.8789130370000002 podStartE2EDuration="6.391733085s" totalImagesPullingTime="3.512820048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:48.390740164 +0000 UTC m=+65.976216416" watchObservedRunningTime="2026-07-17 17:57:48.391733085 +0000 UTC m=+65.977209336"
Jul 17 17:57:48.425622 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.425511    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r26rr" podStartSLOduration=1.931409426 podStartE2EDuration="15.425490716s" totalImagesPullingTime="13.49408129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:57:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:48.425138211 +0000 UTC m=+66.010614466" watchObservedRunningTime="2026-07-17 17:57:48.425490716 +0000 UTC m=+66.010966966"
Jul 17 17:57:48.501769 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.501744    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:48.621471 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.621429    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:48.621956 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.621484    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:48.624199 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.624063    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 17:57:48.624199 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.624082    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 17:57:48.634339 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.634284    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 17:57:48.634816 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.634775    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0c868384-1df1-438e-8e7c-6a7d578a58af-original-pull-secret\") pod \"global-pull-secret-syncer-snwmp\" (UID: \"0c868384-1df1-438e-8e7c-6a7d578a58af\") " pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:48.646176 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.646145    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w2nd\" (UniqueName: \"kubernetes.io/projected/1031a638-0acf-4023-91f8-ba906ca32530-kube-api-access-6w2nd\") pod \"network-check-target-c9xd4\" (UID: \"1031a638-0acf-4023-91f8-ba906ca32530\") " pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:48.647026 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.647005    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"]
Jul 17 17:57:48.649264 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.649239    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jb7qg\""
Jul 17 17:57:48.650461 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:48.650441    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02a87df_bb7d_4ca4_b7e2_0babbac79342.slice/crio-44db174b4e50cf7598e9e6d2808c84bf34d12620735fc021a66522fc453031b7 WatchSource:0}: Error finding container 44db174b4e50cf7598e9e6d2808c84bf34d12620735fc021a66522fc453031b7: Status 404 returned error can't find the container with id 44db174b4e50cf7598e9e6d2808c84bf34d12620735fc021a66522fc453031b7
Jul 17 17:57:48.657573 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.657553    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:48.657682 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.657594    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snwmp"
Jul 17 17:57:48.741660 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.728370    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:48.741660 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.730984    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 17:57:48.745196 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.745132    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6b5f8c0-20a0-4d96-9ab4-b0e256242982-metrics-certs\") pod \"network-metrics-daemon-5n67b\" (UID: \"b6b5f8c0-20a0-4d96-9ab4-b0e256242982\") " pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:48.841832 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.841765    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-snwmp"]
Jul 17 17:57:48.869263 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.869209    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c9xd4"]
Jul 17 17:57:48.873369 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:48.873338    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1031a638_0acf_4023_91f8_ba906ca32530.slice/crio-d081e5d126a656192c6edc41c9465f890a5e3f68229bcb0eb5a8e2f8dda2fcbd WatchSource:0}: Error finding container d081e5d126a656192c6edc41c9465f890a5e3f68229bcb0eb5a8e2f8dda2fcbd: Status 404 returned error can't find the container with id d081e5d126a656192c6edc41c9465f890a5e3f68229bcb0eb5a8e2f8dda2fcbd
Jul 17 17:57:48.955928 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.955902    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dn68c\""
Jul 17 17:57:48.963816 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:48.963790    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5n67b"
Jul 17 17:57:49.118867 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.118784    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5n67b"]
Jul 17 17:57:49.124008 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:49.123966    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b5f8c0_20a0_4d96_9ab4_b0e256242982.slice/crio-a5385ce3361eb2195e2a1cc0e412382cd6c45af9e559827f91861c12cf970cf1 WatchSource:0}: Error finding container a5385ce3361eb2195e2a1cc0e412382cd6c45af9e559827f91861c12cf970cf1: Status 404 returned error can't find the container with id a5385ce3361eb2195e2a1cc0e412382cd6c45af9e559827f91861c12cf970cf1
Jul 17 17:57:49.373947 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.373812    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-snwmp" event={"ID":"0c868384-1df1-438e-8e7c-6a7d578a58af","Type":"ContainerStarted","Data":"5e8e662cb8b9d060c9118ffb19c5be1ae80e98ac866ac6db432c38549b4dcb21"}
Jul 17 17:57:49.379134 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.378166    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ldk6v" event={"ID":"41157244-5641-4575-a240-93418d028431","Type":"ContainerStarted","Data":"a1bd385242fe1b99300db130f4885e8336800c6ed659a869a44303fb866d2b73"}
Jul 17 17:57:49.379134 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.378233    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ldk6v" event={"ID":"41157244-5641-4575-a240-93418d028431","Type":"ContainerStarted","Data":"0610526accfcf69749d561aca03b2db8fe181ae764242931a9e4d64c85dbb93f"}
Jul 17 17:57:49.380909 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.380817    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c9xd4" event={"ID":"1031a638-0acf-4023-91f8-ba906ca32530","Type":"ContainerStarted","Data":"d081e5d126a656192c6edc41c9465f890a5e3f68229bcb0eb5a8e2f8dda2fcbd"}
Jul 17 17:57:49.388181 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.383597    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t" event={"ID":"b02a87df-bb7d-4ca4-b7e2-0babbac79342","Type":"ContainerStarted","Data":"44db174b4e50cf7598e9e6d2808c84bf34d12620735fc021a66522fc453031b7"}
Jul 17 17:57:49.388181 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.386151    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5n67b" event={"ID":"b6b5f8c0-20a0-4d96-9ab4-b0e256242982","Type":"ContainerStarted","Data":"a5385ce3361eb2195e2a1cc0e412382cd6c45af9e559827f91861c12cf970cf1"}
Jul 17 17:57:49.402748 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.402399    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ldk6v" podStartSLOduration=3.644873303 podStartE2EDuration="7.402377615s" totalImagesPullingTime="3.66020462s" totalInitContainerRuntime="97.299692ms" isStatefulPod=true podCreationTimestamp="2026-07-17 17:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:49.402132015 +0000 UTC m=+66.987608271" watchObservedRunningTime="2026-07-17 17:57:49.402377615 +0000 UTC m=+66.987853866"
Jul 17 17:57:49.718459 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.718217    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86d7d47b99-pssnk"]
Jul 17 17:57:49.726918 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.726888    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.732109 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.732027    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d7d47b99-pssnk"]
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733148    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733246    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733403    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733474    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733572    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733690    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733788    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 17:57:49.734119 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.733979    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wgsbz\""
Jul 17 17:57:49.843091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.842782    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-oauth-serving-cert\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.843091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.842841    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbqw4\" (UniqueName: \"kubernetes.io/projected/27a710af-afd9-48ab-bcee-f60a4ead23dc-kube-api-access-fbqw4\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.843091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.842882    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-oauth-config\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.843091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.842923    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-serving-cert\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.843091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.842953    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-config\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.843091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.842987    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-service-ca\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.944492 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.944369    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbqw4\" (UniqueName: \"kubernetes.io/projected/27a710af-afd9-48ab-bcee-f60a4ead23dc-kube-api-access-fbqw4\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.944492 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.944427    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-oauth-config\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.944492 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.944467    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-serving-cert\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.944492 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.944498    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-config\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.944886 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.944529    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-service-ca\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.944886 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.944576    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-oauth-serving-cert\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.945455 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.945407    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-oauth-serving-cert\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.945820 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.945796    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-service-ca\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.946081 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.946002    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-config\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.948967 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.948925    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-oauth-config\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.950919 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.950878    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-serving-cert\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:49.952914 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:49.952870    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbqw4\" (UniqueName: \"kubernetes.io/projected/27a710af-afd9-48ab-bcee-f60a4ead23dc-kube-api-access-fbqw4\") pod \"console-86d7d47b99-pssnk\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") " pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:50.046411 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:50.046153    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:57:50.319252 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:50.319168    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nmplq"
Jul 17 17:57:51.223268 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.223244    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d7d47b99-pssnk"]
Jul 17 17:57:51.395134 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.395100    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5n67b" event={"ID":"b6b5f8c0-20a0-4d96-9ab4-b0e256242982","Type":"ContainerStarted","Data":"fec14eaa54b626a4696452c32fa51da99ada55faf459231d35e200014141f863"}
Jul 17 17:57:51.396443 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.396417    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d7d47b99-pssnk" event={"ID":"27a710af-afd9-48ab-bcee-f60a4ead23dc","Type":"ContainerStarted","Data":"6633f14c19cc65e261c811134ed39162e1efa454b7c77a2cfbca8a75cdc51e49"}
Jul 17 17:57:51.398127 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.398059    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t" event={"ID":"b02a87df-bb7d-4ca4-b7e2-0babbac79342","Type":"ContainerStarted","Data":"d606416bc108f332247ffa831b7fc3123549e20cd5954268043f6d10bf70922f"}
Jul 17 17:57:51.398127 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.398104    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:51.402046 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.401880    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" event={"ID":"8dd1f855-6db3-4d9f-935e-9dfa0e925e65","Type":"ContainerStarted","Data":"16884cc2aa283ed93aba1a10532e1f9fcde2f36d324c10187a4797995277bf42"}
Jul 17 17:57:51.402046 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.401951    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" event={"ID":"8dd1f855-6db3-4d9f-935e-9dfa0e925e65","Type":"ContainerStarted","Data":"ca28bfb4144202758a261beeee1830ed5326858ce8afd448fff71312243ad714"}
Jul 17 17:57:51.402046 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.401990    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" event={"ID":"8dd1f855-6db3-4d9f-935e-9dfa0e925e65","Type":"ContainerStarted","Data":"404e1157f344b424bcb4e517646b89d6ecb33308e20f5206d9b61af276198797"}
Jul 17 17:57:51.405326 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.405215    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t"
Jul 17 17:57:51.430733 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:51.430551    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-z9w7t" podStartSLOduration=2.012830952 podStartE2EDuration="4.430532123s" totalImagesPullingTime="2.417701171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:51.414702503 +0000 UTC m=+69.000178758" watchObservedRunningTime="2026-07-17 17:57:51.430532123 +0000 UTC m=+69.016008353"
Jul 17 17:57:52.411674 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:52.411619    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5n67b" event={"ID":"b6b5f8c0-20a0-4d96-9ab4-b0e256242982","Type":"ContainerStarted","Data":"724baa71a5a523f03a1f0a6b18191091b431d614b546fc82e66887784acc0e8f"}
Jul 17 17:57:52.427101 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:52.427052    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5n67b" podStartSLOduration=67.419750503 podStartE2EDuration="1m9.42703181s" totalImagesPullingTime="2.007281307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:52.426839679 +0000 UTC m=+70.012315952" watchObservedRunningTime="2026-07-17 17:57:52.42703181 +0000 UTC m=+70.012508061"
Jul 17 17:57:56.429950 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.429918    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-snwmp" event={"ID":"0c868384-1df1-438e-8e7c-6a7d578a58af","Type":"ContainerStarted","Data":"a55cc1b4621d5f2d9faea930e2cc42fe86c51da279164330df95f62dd59fbee0"}
Jul 17 17:57:56.431528 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.431503    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d7d47b99-pssnk" event={"ID":"27a710af-afd9-48ab-bcee-f60a4ead23dc","Type":"ContainerStarted","Data":"5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8"}
Jul 17 17:57:56.433100 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.433077    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c9xd4" event={"ID":"1031a638-0acf-4023-91f8-ba906ca32530","Type":"ContainerStarted","Data":"a219dd82c497053cdb5972041357e7680a7cc28ad5f757575f9db56d43a9375a"}
Jul 17 17:57:56.433210 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.433167    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:57:56.435928 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.435909    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" event={"ID":"8dd1f855-6db3-4d9f-935e-9dfa0e925e65","Type":"ContainerStarted","Data":"345692e2d75d91b770085ad34c5cd2d658272b7fb733bd68fa86b4d2d5361c72"}
Jul 17 17:57:56.436003 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.435956    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" event={"ID":"8dd1f855-6db3-4d9f-935e-9dfa0e925e65","Type":"ContainerStarted","Data":"6d1ce9b32b152737fca085179e9faf0fd68accefc68979c1f9f2a9fc2ffd6c80"}
Jul 17 17:57:56.436003 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.435987    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" event={"ID":"8dd1f855-6db3-4d9f-935e-9dfa0e925e65","Type":"ContainerStarted","Data":"fe1d2fb00736a2d61deec33368c0e4192de2d8c8a3d34ab68d8a591cfb81f15d"}
Jul 17 17:57:56.436078 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.436017    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:56.441619 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.441600    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f"
Jul 17 17:57:56.445688 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.445617    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-snwmp" podStartSLOduration=66.598023127 podStartE2EDuration="1m13.4456031s" totalImagesPullingTime="6.847579973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:56.444553931 +0000 UTC m=+74.030030186" watchObservedRunningTime="2026-07-17 17:57:56.4456031 +0000 UTC m=+74.031079354"
Jul 17 17:57:56.470428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.470365    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6cfb69d5b5-cfl9f" podStartSLOduration=3.682107383 podStartE2EDuration="11.470342981s" totalImagesPullingTime="7.788235598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:56.468748857 +0000 UTC m=+74.054225130" watchObservedRunningTime="2026-07-17 17:57:56.470342981 +0000 UTC m=+74.055819234"
Jul 17 17:57:56.485510 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.485459    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c9xd4" podStartSLOduration=66.665428606 podStartE2EDuration="1m13.485444431s" totalImagesPullingTime="6.820015825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:56.484968093 +0000 UTC m=+74.070444346" watchObservedRunningTime="2026-07-17 17:57:56.485444431 +0000 UTC m=+74.070920682"
Jul 17 17:57:56.502790 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:56.502734    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86d7d47b99-pssnk" podStartSLOduration=3.029787133 podStartE2EDuration="7.502718619s" totalImagesPullingTime="4.472931486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:57:56.501916037 +0000 UTC m=+74.087392289" watchObservedRunningTime="2026-07-17 17:57:56.502718619 +0000 UTC m=+74.088194887"
Jul 17 17:57:59.384984 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.384948    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b96959d84-mj6r4"]
Jul 17 17:57:59.388293 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.388275    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.399582 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.399544    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b96959d84-mj6r4"]
Jul 17 17:57:59.401497 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.401475    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 17:57:59.541817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.541787    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2thvc\" (UniqueName: \"kubernetes.io/projected/c14e5665-4a2e-4589-a4e7-2fb414c44298-kube-api-access-2thvc\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.541817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.541819    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-service-ca\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.542035 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.541847    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-trusted-ca-bundle\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.542035 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.541917    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-config\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.542035 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.541960    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-serving-cert\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.542035 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.541982    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-oauth-serving-cert\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.542035 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.542014    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-oauth-config\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.642710 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.642617    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-serving-cert\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.642710 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.642679    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-oauth-serving-cert\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.642710 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.642709    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-oauth-config\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.642972 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.642739    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2thvc\" (UniqueName: \"kubernetes.io/projected/c14e5665-4a2e-4589-a4e7-2fb414c44298-kube-api-access-2thvc\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.642972 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.642757    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-service-ca\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.642972 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.642774    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-trusted-ca-bundle\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.642972 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.642867    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-config\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.643570 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.643542    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-oauth-serving-cert\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.643727 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.643687    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-config\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.643727 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.643690    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-service-ca\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.643837 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.643783    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-trusted-ca-bundle\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.645093 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.645076    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-oauth-config\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.645232 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.645215    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-serving-cert\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.650398 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.650376    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2thvc\" (UniqueName: \"kubernetes.io/projected/c14e5665-4a2e-4589-a4e7-2fb414c44298-kube-api-access-2thvc\") pod \"console-b96959d84-mj6r4\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") " pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.699143 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.699109    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:57:59.834690 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:57:59.834666    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b96959d84-mj6r4"]
Jul 17 17:57:59.837199 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:57:59.837172    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14e5665_4a2e_4589_a4e7_2fb414c44298.slice/crio-066b4c39fb26a1793024900e0e2be8e0e68c32f881db8ae2d6c896252734d6f9 WatchSource:0}: Error finding container 066b4c39fb26a1793024900e0e2be8e0e68c32f881db8ae2d6c896252734d6f9: Status 404 returned error can't find the container with id 066b4c39fb26a1793024900e0e2be8e0e68c32f881db8ae2d6c896252734d6f9
Jul 17 17:58:00.047059 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:00.047027    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:58:00.047223 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:00.047074    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:58:00.051997 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:00.051971    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:58:00.456318 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:00.456283    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b96959d84-mj6r4" event={"ID":"c14e5665-4a2e-4589-a4e7-2fb414c44298","Type":"ContainerStarted","Data":"f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96"}
Jul 17 17:58:00.456808 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:00.456338    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b96959d84-mj6r4" event={"ID":"c14e5665-4a2e-4589-a4e7-2fb414c44298","Type":"ContainerStarted","Data":"066b4c39fb26a1793024900e0e2be8e0e68c32f881db8ae2d6c896252734d6f9"}
Jul 17 17:58:00.460477 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:00.460457    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:58:00.474881 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:00.474829    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b96959d84-mj6r4" podStartSLOduration=1.474813418 podStartE2EDuration="1.474813418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:58:00.474187604 +0000 UTC m=+78.059663857" watchObservedRunningTime="2026-07-17 17:58:00.474813418 +0000 UTC m=+78.060289670"
Jul 17 17:58:09.699788 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:09.699669    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:58:09.699788 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:09.699726    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:58:09.704459 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:09.704433    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:58:10.496791 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:10.496765    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:58:10.544134 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:10.544104    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d7d47b99-pssnk"]
Jul 17 17:58:27.442888 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:27.442852    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c9xd4"
Jul 17 17:58:35.565104 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.565057    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86d7d47b99-pssnk" podUID="27a710af-afd9-48ab-bcee-f60a4ead23dc" containerName="console" containerID="cri-o://5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8" gracePeriod=15
Jul 17 17:58:35.802370 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.802349    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:58:35.813764 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.813742    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-oauth-config\") pod \"27a710af-afd9-48ab-bcee-f60a4ead23dc\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") "
Jul 17 17:58:35.813868 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.813830    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-oauth-serving-cert\") pod \"27a710af-afd9-48ab-bcee-f60a4ead23dc\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") "
Jul 17 17:58:35.813913 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.813886    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-serving-cert\") pod \"27a710af-afd9-48ab-bcee-f60a4ead23dc\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") "
Jul 17 17:58:35.813958 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.813924    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-config\") pod \"27a710af-afd9-48ab-bcee-f60a4ead23dc\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") "
Jul 17 17:58:35.814011 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.813956    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbqw4\" (UniqueName: \"kubernetes.io/projected/27a710af-afd9-48ab-bcee-f60a4ead23dc-kube-api-access-fbqw4\") pod \"27a710af-afd9-48ab-bcee-f60a4ead23dc\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") "
Jul 17 17:58:35.814011 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.813985    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-service-ca\") pod \"27a710af-afd9-48ab-bcee-f60a4ead23dc\" (UID: \"27a710af-afd9-48ab-bcee-f60a4ead23dc\") "
Jul 17 17:58:35.814257 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.814230    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "27a710af-afd9-48ab-bcee-f60a4ead23dc" (UID: "27a710af-afd9-48ab-bcee-f60a4ead23dc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:58:35.814378 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.814344    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-config" (OuterVolumeSpecName: "console-config") pod "27a710af-afd9-48ab-bcee-f60a4ead23dc" (UID: "27a710af-afd9-48ab-bcee-f60a4ead23dc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:58:35.814484 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.814389    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-service-ca" (OuterVolumeSpecName: "service-ca") pod "27a710af-afd9-48ab-bcee-f60a4ead23dc" (UID: "27a710af-afd9-48ab-bcee-f60a4ead23dc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:58:35.816125 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.816067    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "27a710af-afd9-48ab-bcee-f60a4ead23dc" (UID: "27a710af-afd9-48ab-bcee-f60a4ead23dc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 17:58:35.816201 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.816165    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "27a710af-afd9-48ab-bcee-f60a4ead23dc" (UID: "27a710af-afd9-48ab-bcee-f60a4ead23dc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 17:58:35.816201 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.816177    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a710af-afd9-48ab-bcee-f60a4ead23dc-kube-api-access-fbqw4" (OuterVolumeSpecName: "kube-api-access-fbqw4") pod "27a710af-afd9-48ab-bcee-f60a4ead23dc" (UID: "27a710af-afd9-48ab-bcee-f60a4ead23dc"). InnerVolumeSpecName "kube-api-access-fbqw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:58:35.914546 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.914511    2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-oauth-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:58:35.914546 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.914541    2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:58:35.914546 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.914552    2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:58:35.914817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.914560    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbqw4\" (UniqueName: \"kubernetes.io/projected/27a710af-afd9-48ab-bcee-f60a4ead23dc-kube-api-access-fbqw4\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:58:35.914817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.914570    2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27a710af-afd9-48ab-bcee-f60a4ead23dc-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:58:35.914817 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:35.914578    2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27a710af-afd9-48ab-bcee-f60a4ead23dc-console-oauth-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:58:36.607922 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.607890    2555 generic.go:356] "Generic (PLEG): container finished" podID="27a710af-afd9-48ab-bcee-f60a4ead23dc" containerID="5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8" exitCode=0
Jul 17 17:58:36.608335 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.607930    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d7d47b99-pssnk"
Jul 17 17:58:36.608335 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.607982    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d7d47b99-pssnk" event={"ID":"27a710af-afd9-48ab-bcee-f60a4ead23dc","Type":"ContainerDied","Data":"5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8"}
Jul 17 17:58:36.608335 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.608038    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d7d47b99-pssnk" event={"ID":"27a710af-afd9-48ab-bcee-f60a4ead23dc","Type":"ContainerDied","Data":"6633f14c19cc65e261c811134ed39162e1efa454b7c77a2cfbca8a75cdc51e49"}
Jul 17 17:58:36.608335 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.608062    2555 scope.go:117] "RemoveContainer" containerID="5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8"
Jul 17 17:58:36.617013 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.616993    2555 scope.go:117] "RemoveContainer" containerID="5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8"
Jul 17 17:58:36.617268 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:58:36.617249    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8\": container with ID starting with 5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8 not found: ID does not exist" containerID="5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8"
Jul 17 17:58:36.617311 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.617278    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8"} err="failed to get container status \"5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8\": rpc error: code = NotFound desc = could not find container \"5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8\": container with ID starting with 5dc411d20a69fe4e29e016aaab2a92ebe8ea4638cfe88fdbcd8e28abf6f91bc8 not found: ID does not exist"
Jul 17 17:58:36.629696 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.629675    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d7d47b99-pssnk"]
Jul 17 17:58:36.632998 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.632976    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86d7d47b99-pssnk"]
Jul 17 17:58:36.941950 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:58:36.941872    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27a710af-afd9-48ab-bcee-f60a4ead23dc" path="/var/lib/kubelet/pods/27a710af-afd9-48ab-bcee-f60a4ead23dc/volumes"
Jul 17 17:59:05.009925 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.009894    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 17:59:05.010528 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.010161    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a710af-afd9-48ab-bcee-f60a4ead23dc" containerName="console"
Jul 17 17:59:05.010528 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.010174    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a710af-afd9-48ab-bcee-f60a4ead23dc" containerName="console"
Jul 17 17:59:05.010528 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.010228    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="27a710af-afd9-48ab-bcee-f60a4ead23dc" containerName="console"
Jul 17 17:59:05.013487 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.013468    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.015863 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.015844    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 17 17:59:05.015975 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.015864    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gc28g\""
Jul 17 17:59:05.015975 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.015940    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 17 17:59:05.016139 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.016126    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 17 17:59:05.016334 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.016322    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 17 17:59:05.016651 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.016613    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 17 17:59:05.016761 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.016732    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 17 17:59:05.016824 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.016783    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 17 17:59:05.017193 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.017165    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 17 17:59:05.023083 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.023064    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 17 17:59:05.031101 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.031077    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 17:59:05.130036 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130003    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130041    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130065    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130083    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130127    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tf2v\" (UniqueName: \"kubernetes.io/projected/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-kube-api-access-7tf2v\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130162    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130183    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130213 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130201    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130240    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130276    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130307    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-web-config\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130323    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.130428 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.130343    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-config-out\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231173 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231124    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-config-out\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231173 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231182    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231210    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231237    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231260    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231283    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tf2v\" (UniqueName: \"kubernetes.io/projected/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-kube-api-access-7tf2v\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231309    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231336    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231360    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231445 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231412    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231864 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231446    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231864 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231489    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-web-config\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231864 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231513    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.231864 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.231782    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.232308 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.232283    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.234232 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.234096    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-config-out\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.234555 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.234348    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.234555 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.234400    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.234555 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.234542    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.234741 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.234547    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.234849 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.234830    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.235026 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.235005    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-web-config\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.235249 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.235229    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.235320 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.235302    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.236272 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.236252    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.239149 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.239132    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tf2v\" (UniqueName: \"kubernetes.io/projected/dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2-kube-api-access-7tf2v\") pod \"alertmanager-main-0\" (UID: \"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.324674 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.324581    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 17:59:05.460121 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.460096    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 17:59:05.462180 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:59:05.462157    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5f1227_6a6f_4afc_969b_4ea9a04ae5c2.slice/crio-334d07a7f18dbb89dca2a81d9f5cbd28030a418a6d26368a4b178a8fdd1211e1 WatchSource:0}: Error finding container 334d07a7f18dbb89dca2a81d9f5cbd28030a418a6d26368a4b178a8fdd1211e1: Status 404 returned error can't find the container with id 334d07a7f18dbb89dca2a81d9f5cbd28030a418a6d26368a4b178a8fdd1211e1
Jul 17 17:59:05.713138 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:05.713051    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerStarted","Data":"334d07a7f18dbb89dca2a81d9f5cbd28030a418a6d26368a4b178a8fdd1211e1"}
Jul 17 17:59:07.009138 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.006268    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-846f4ff7b6-m6v4l"]
Jul 17 17:59:07.010527 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.010508    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.019841 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.019817    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846f4ff7b6-m6v4l"]
Jul 17 17:59:07.148265 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.148227    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk4xv\" (UniqueName: \"kubernetes.io/projected/8aa9b44c-6a8a-4114-9a03-52ee1e053308-kube-api-access-wk4xv\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.148265 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.148274    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-service-ca\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.148671 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.148361    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-trusted-ca-bundle\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.148671 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.148413    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-oauth-config\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.148671 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.148449    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-oauth-serving-cert\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.148671 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.148525    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-config\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.148671 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.148573    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-serving-cert\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.249715 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.249674    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-config\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.249993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.249800    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-serving-cert\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.249993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.249834    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk4xv\" (UniqueName: \"kubernetes.io/projected/8aa9b44c-6a8a-4114-9a03-52ee1e053308-kube-api-access-wk4xv\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.249993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.249870    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-service-ca\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.249993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.249910    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-trusted-ca-bundle\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.249993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.249947    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-oauth-config\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.249993 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.249973    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-oauth-serving-cert\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.250497 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.250468    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-config\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.250713 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.250687    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-service-ca\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.250768 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.250697    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-oauth-serving-cert\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.250986 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.250958    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-trusted-ca-bundle\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.253225 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.253201    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-serving-cert\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.253387 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.253255    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-oauth-config\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.258030 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.258003    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk4xv\" (UniqueName: \"kubernetes.io/projected/8aa9b44c-6a8a-4114-9a03-52ee1e053308-kube-api-access-wk4xv\") pod \"console-846f4ff7b6-m6v4l\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") " pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.321104 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.321010    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:07.497245 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.497222    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846f4ff7b6-m6v4l"]
Jul 17 17:59:07.499572 ip-10-0-140-23 kubenswrapper[2555]: W0717 17:59:07.499547    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa9b44c_6a8a_4114_9a03_52ee1e053308.slice/crio-c205b554ee742bbac2b235899753dcc198147a0318723ff6334bbc6d7a84f875 WatchSource:0}: Error finding container c205b554ee742bbac2b235899753dcc198147a0318723ff6334bbc6d7a84f875: Status 404 returned error can't find the container with id c205b554ee742bbac2b235899753dcc198147a0318723ff6334bbc6d7a84f875
Jul 17 17:59:07.721676 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.721576    2555 generic.go:356] "Generic (PLEG): container finished" podID="dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2" containerID="48a6a60e626a6469632b3f1b11adedd81839ca0675f2bf1f248b102ef40a98f7" exitCode=0
Jul 17 17:59:07.721826 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.721666    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerDied","Data":"48a6a60e626a6469632b3f1b11adedd81839ca0675f2bf1f248b102ef40a98f7"}
Jul 17 17:59:07.723261 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.723230    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846f4ff7b6-m6v4l" event={"ID":"8aa9b44c-6a8a-4114-9a03-52ee1e053308","Type":"ContainerStarted","Data":"3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4"}
Jul 17 17:59:07.723347 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.723273    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846f4ff7b6-m6v4l" event={"ID":"8aa9b44c-6a8a-4114-9a03-52ee1e053308","Type":"ContainerStarted","Data":"c205b554ee742bbac2b235899753dcc198147a0318723ff6334bbc6d7a84f875"}
Jul 17 17:59:07.760599 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:07.760540    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-846f4ff7b6-m6v4l" podStartSLOduration=1.760522692 podStartE2EDuration="1.760522692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 17:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:59:07.760033319 +0000 UTC m=+145.345509575" watchObservedRunningTime="2026-07-17 17:59:07.760522692 +0000 UTC m=+145.345998944"
Jul 17 17:59:09.733165 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:09.733065    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerStarted","Data":"bb7e95621ac6e8283b01688aed51b4d1290b91a178b00b8aaa136aca0febef6c"}
Jul 17 17:59:09.733165 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:09.733144    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerStarted","Data":"df5546c55fecc8f4230d2001aef87b04c75aeb445fb86cd0c59646bea5653495"}
Jul 17 17:59:09.733749 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:09.733184    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerStarted","Data":"f38bc6d6ae4bf9894e1933c1ce5638a27fde04ee8b21923c582d87f44224ee63"}
Jul 17 17:59:09.733749 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:09.733226    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerStarted","Data":"0333e2ca8eb1e975e15aa2b2b77c17e9bdf9fa7adbdcab5578706f44008931e2"}
Jul 17 17:59:09.733749 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:09.733263    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerStarted","Data":"ad6635dae0d756e24f92f6dc8503ecd4e2d9622621bac14f57107a1152fd0331"}
Jul 17 17:59:09.733749 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:09.733303    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2","Type":"ContainerStarted","Data":"a3eee15c7a887ef638ffc41cc39428c1457d4020e99e108d75633349c52ef37a"}
Jul 17 17:59:09.762162 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:09.762090    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.145945016 podStartE2EDuration="5.762068589s" totalImagesPullingTime="2.600813065s" totalInitContainerRuntime="15.310508ms" isStatefulPod=false podCreationTimestamp="2026-07-17 17:59:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 17:59:09.760490277 +0000 UTC m=+147.345966525" watchObservedRunningTime="2026-07-17 17:59:09.762068589 +0000 UTC m=+147.347544843"
Jul 17 17:59:17.321930 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:17.321900    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:17.322345 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:17.321980    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:17.326627 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:17.326604    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:17.765593 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:17.765566    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 17:59:17.847854 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:17.847826    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b96959d84-mj6r4"]
Jul 17 17:59:42.867859 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:42.867819    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b96959d84-mj6r4" podUID="c14e5665-4a2e-4589-a4e7-2fb414c44298" containerName="console" containerID="cri-o://f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96" gracePeriod=15
Jul 17 17:59:43.110296 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.110272    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:59:43.150966 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.150870    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-service-ca\") pod \"c14e5665-4a2e-4589-a4e7-2fb414c44298\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") "
Jul 17 17:59:43.150966 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.150921    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-oauth-config\") pod \"c14e5665-4a2e-4589-a4e7-2fb414c44298\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") "
Jul 17 17:59:43.150966 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.150941    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-trusted-ca-bundle\") pod \"c14e5665-4a2e-4589-a4e7-2fb414c44298\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") "
Jul 17 17:59:43.151244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.150986    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2thvc\" (UniqueName: \"kubernetes.io/projected/c14e5665-4a2e-4589-a4e7-2fb414c44298-kube-api-access-2thvc\") pod \"c14e5665-4a2e-4589-a4e7-2fb414c44298\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") "
Jul 17 17:59:43.151244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.151043    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-oauth-serving-cert\") pod \"c14e5665-4a2e-4589-a4e7-2fb414c44298\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") "
Jul 17 17:59:43.151244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.151071    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-config\") pod \"c14e5665-4a2e-4589-a4e7-2fb414c44298\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") "
Jul 17 17:59:43.151244 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.151138    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-serving-cert\") pod \"c14e5665-4a2e-4589-a4e7-2fb414c44298\" (UID: \"c14e5665-4a2e-4589-a4e7-2fb414c44298\") "
Jul 17 17:59:43.151452 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.151288    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-service-ca" (OuterVolumeSpecName: "service-ca") pod "c14e5665-4a2e-4589-a4e7-2fb414c44298" (UID: "c14e5665-4a2e-4589-a4e7-2fb414c44298"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:59:43.151504 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.151484    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c14e5665-4a2e-4589-a4e7-2fb414c44298" (UID: "c14e5665-4a2e-4589-a4e7-2fb414c44298"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:59:43.151558 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.151513    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c14e5665-4a2e-4589-a4e7-2fb414c44298" (UID: "c14e5665-4a2e-4589-a4e7-2fb414c44298"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:59:43.151620 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.151595    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-config" (OuterVolumeSpecName: "console-config") pod "c14e5665-4a2e-4589-a4e7-2fb414c44298" (UID: "c14e5665-4a2e-4589-a4e7-2fb414c44298"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 17:59:43.153091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.153063    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c14e5665-4a2e-4589-a4e7-2fb414c44298" (UID: "c14e5665-4a2e-4589-a4e7-2fb414c44298"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 17:59:43.153091 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.153064    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14e5665-4a2e-4589-a4e7-2fb414c44298-kube-api-access-2thvc" (OuterVolumeSpecName: "kube-api-access-2thvc") pod "c14e5665-4a2e-4589-a4e7-2fb414c44298" (UID: "c14e5665-4a2e-4589-a4e7-2fb414c44298"). InnerVolumeSpecName "kube-api-access-2thvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 17:59:43.153264 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.153128    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c14e5665-4a2e-4589-a4e7-2fb414c44298" (UID: "c14e5665-4a2e-4589-a4e7-2fb414c44298"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 17:59:43.252473 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.252422    2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:59:43.252473 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.252468    2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:59:43.252473 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.252478    2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-oauth-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:59:43.252473 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.252486    2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-trusted-ca-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:59:43.252473 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.252495    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2thvc\" (UniqueName: \"kubernetes.io/projected/c14e5665-4a2e-4589-a4e7-2fb414c44298-kube-api-access-2thvc\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:59:43.252793 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.252503    2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-oauth-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:59:43.252793 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.252511    2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c14e5665-4a2e-4589-a4e7-2fb414c44298-console-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 17:59:43.867882 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.867848    2555 generic.go:356] "Generic (PLEG): container finished" podID="c14e5665-4a2e-4589-a4e7-2fb414c44298" containerID="f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96" exitCode=0
Jul 17 17:59:43.867882 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.867888    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b96959d84-mj6r4" event={"ID":"c14e5665-4a2e-4589-a4e7-2fb414c44298","Type":"ContainerDied","Data":"f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96"}
Jul 17 17:59:43.868365 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.867900    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b96959d84-mj6r4"
Jul 17 17:59:43.868365 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.867923    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b96959d84-mj6r4" event={"ID":"c14e5665-4a2e-4589-a4e7-2fb414c44298","Type":"ContainerDied","Data":"066b4c39fb26a1793024900e0e2be8e0e68c32f881db8ae2d6c896252734d6f9"}
Jul 17 17:59:43.868365 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.867942    2555 scope.go:117] "RemoveContainer" containerID="f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96"
Jul 17 17:59:43.878407 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.878361    2555 scope.go:117] "RemoveContainer" containerID="f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96"
Jul 17 17:59:43.879000 ip-10-0-140-23 kubenswrapper[2555]: E0717 17:59:43.878976    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96\": container with ID starting with f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96 not found: ID does not exist" containerID="f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96"
Jul 17 17:59:43.879105 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.879006    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96"} err="failed to get container status \"f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96\": rpc error: code = NotFound desc = could not find container \"f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96\": container with ID starting with f8963737e429f0a07ef24047e1a4918c4d5f26340aa8732bca333e8d33800a96 not found: ID does not exist"
Jul 17 17:59:43.891851 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.891825    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b96959d84-mj6r4"]
Jul 17 17:59:43.895724 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:43.895700    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b96959d84-mj6r4"]
Jul 17 17:59:44.942661 ip-10-0-140-23 kubenswrapper[2555]: I0717 17:59:44.942609    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14e5665-4a2e-4589-a4e7-2fb414c44298" path="/var/lib/kubelet/pods/c14e5665-4a2e-4589-a4e7-2fb414c44298/volumes"
Jul 17 18:00:06.236875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.236839    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"]
Jul 17 18:00:06.237298 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.237142    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14e5665-4a2e-4589-a4e7-2fb414c44298" containerName="console"
Jul 17 18:00:06.237298 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.237152    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14e5665-4a2e-4589-a4e7-2fb414c44298" containerName="console"
Jul 17 18:00:06.237298 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.237238    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14e5665-4a2e-4589-a4e7-2fb414c44298" containerName="console"
Jul 17 18:00:06.240117 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.240101    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.242662 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.242620    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 18:00:06.242793 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.242753    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 18:00:06.243481 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.243461    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m5qws\""
Jul 17 18:00:06.249117 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.249092    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"]
Jul 17 18:00:06.343584 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.343545    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5vst\" (UniqueName: \"kubernetes.io/projected/c8171c29-e578-4741-9504-de4c5b137b0d-kube-api-access-n5vst\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.343793 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.343600    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.343793 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.343661    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.445635 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.445592    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5vst\" (UniqueName: \"kubernetes.io/projected/c8171c29-e578-4741-9504-de4c5b137b0d-kube-api-access-n5vst\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.445836 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.445669    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.445836 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.445701    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.446085 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.446069    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.446375 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.446353    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.455248 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.455221    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5vst\" (UniqueName: \"kubernetes.io/projected/c8171c29-e578-4741-9504-de4c5b137b0d-kube-api-access-n5vst\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.550055 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.550005    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:06.679995 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.679962    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"]
Jul 17 18:00:06.682710 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:00:06.682679    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8171c29_e578_4741_9504_de4c5b137b0d.slice/crio-a58eee643e1241f3faf509837547abb9634b40d0d32123eac3cde69f6b493e9f WatchSource:0}: Error finding container a58eee643e1241f3faf509837547abb9634b40d0d32123eac3cde69f6b493e9f: Status 404 returned error can't find the container with id a58eee643e1241f3faf509837547abb9634b40d0d32123eac3cde69f6b493e9f
Jul 17 18:00:06.954704 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:06.954618    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt" event={"ID":"c8171c29-e578-4741-9504-de4c5b137b0d","Type":"ContainerStarted","Data":"a58eee643e1241f3faf509837547abb9634b40d0d32123eac3cde69f6b493e9f"}
Jul 17 18:00:14.987378 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:14.987338    2555 generic.go:356] "Generic (PLEG): container finished" podID="c8171c29-e578-4741-9504-de4c5b137b0d" containerID="f8aa5a2c32646f5c332460a0373bdb0a5b784d3ccdf4f2947eec7652ff528ba9" exitCode=0
Jul 17 18:00:14.987793 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:14.987412    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt" event={"ID":"c8171c29-e578-4741-9504-de4c5b137b0d","Type":"ContainerDied","Data":"f8aa5a2c32646f5c332460a0373bdb0a5b784d3ccdf4f2947eec7652ff528ba9"}
Jul 17 18:00:18.000943 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:18.000907    2555 generic.go:356] "Generic (PLEG): container finished" podID="c8171c29-e578-4741-9504-de4c5b137b0d" containerID="d84d90a238ed32f3c069f823fa89ad35993a06c8d42f17a63a9c190bc26d0d6a" exitCode=0
Jul 17 18:00:18.001378 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:18.000989    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt" event={"ID":"c8171c29-e578-4741-9504-de4c5b137b0d","Type":"ContainerDied","Data":"d84d90a238ed32f3c069f823fa89ad35993a06c8d42f17a63a9c190bc26d0d6a"}
Jul 17 18:00:26.036176 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:26.036140    2555 generic.go:356] "Generic (PLEG): container finished" podID="c8171c29-e578-4741-9504-de4c5b137b0d" containerID="fb1b55e657fbdf037973b4eebadd4f361ca6d75a7d04ddb3715a28f25d83c4cc" exitCode=0
Jul 17 18:00:26.036577 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:26.036184    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt" event={"ID":"c8171c29-e578-4741-9504-de4c5b137b0d","Type":"ContainerDied","Data":"fb1b55e657fbdf037973b4eebadd4f361ca6d75a7d04ddb3715a28f25d83c4cc"}
Jul 17 18:00:27.163002 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.162971    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:27.217770 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.217733    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-util\") pod \"c8171c29-e578-4741-9504-de4c5b137b0d\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") "
Jul 17 18:00:27.217955 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.217781    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5vst\" (UniqueName: \"kubernetes.io/projected/c8171c29-e578-4741-9504-de4c5b137b0d-kube-api-access-n5vst\") pod \"c8171c29-e578-4741-9504-de4c5b137b0d\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") "
Jul 17 18:00:27.217955 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.217829    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-bundle\") pod \"c8171c29-e578-4741-9504-de4c5b137b0d\" (UID: \"c8171c29-e578-4741-9504-de4c5b137b0d\") "
Jul 17 18:00:27.218508 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.218483    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-bundle" (OuterVolumeSpecName: "bundle") pod "c8171c29-e578-4741-9504-de4c5b137b0d" (UID: "c8171c29-e578-4741-9504-de4c5b137b0d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:00:27.220055 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.220027    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8171c29-e578-4741-9504-de4c5b137b0d-kube-api-access-n5vst" (OuterVolumeSpecName: "kube-api-access-n5vst") pod "c8171c29-e578-4741-9504-de4c5b137b0d" (UID: "c8171c29-e578-4741-9504-de4c5b137b0d"). InnerVolumeSpecName "kube-api-access-n5vst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:00:27.222115 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.222090    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-util" (OuterVolumeSpecName: "util") pod "c8171c29-e578-4741-9504-de4c5b137b0d" (UID: "c8171c29-e578-4741-9504-de4c5b137b0d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:00:27.322744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.319254    2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:00:27.322744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.319305    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5vst\" (UniqueName: \"kubernetes.io/projected/c8171c29-e578-4741-9504-de4c5b137b0d-kube-api-access-n5vst\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:00:27.322744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:27.319324    2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8171c29-e578-4741-9504-de4c5b137b0d-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:00:28.045273 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:28.045238    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt" event={"ID":"c8171c29-e578-4741-9504-de4c5b137b0d","Type":"ContainerDied","Data":"a58eee643e1241f3faf509837547abb9634b40d0d32123eac3cde69f6b493e9f"}
Jul 17 18:00:28.045444 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:28.045288    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a58eee643e1241f3faf509837547abb9634b40d0d32123eac3cde69f6b493e9f"
Jul 17 18:00:28.045444 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:28.045293    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mqxmt"
Jul 17 18:00:34.304807 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.304778    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"]
Jul 17 18:00:34.305213 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.305063    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8171c29-e578-4741-9504-de4c5b137b0d" containerName="extract"
Jul 17 18:00:34.305213 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.305073    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8171c29-e578-4741-9504-de4c5b137b0d" containerName="extract"
Jul 17 18:00:34.305213 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.305091    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8171c29-e578-4741-9504-de4c5b137b0d" containerName="util"
Jul 17 18:00:34.305213 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.305097    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8171c29-e578-4741-9504-de4c5b137b0d" containerName="util"
Jul 17 18:00:34.305213 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.305106    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8171c29-e578-4741-9504-de4c5b137b0d" containerName="pull"
Jul 17 18:00:34.305213 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.305111    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8171c29-e578-4741-9504-de4c5b137b0d" containerName="pull"
Jul 17 18:00:34.305213 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.305156    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8171c29-e578-4741-9504-de4c5b137b0d" containerName="extract"
Jul 17 18:00:34.308071 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.308054    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.310504 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.310477    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 17 18:00:34.310631 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.310585    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 17 18:00:34.310631 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.310595    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-n6wll\""
Jul 17 18:00:34.320889 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.320863    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"]
Jul 17 18:00:34.368747 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.368708    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-892ds\" (UniqueName: \"kubernetes.io/projected/51a8af69-0b4c-4247-998c-16224795d2a2-kube-api-access-892ds\") pod \"cert-manager-operator-controller-manager-55c444bf86-xtf4d\" (UID: \"51a8af69-0b4c-4247-998c-16224795d2a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.368946 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.368779    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8af69-0b4c-4247-998c-16224795d2a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xtf4d\" (UID: \"51a8af69-0b4c-4247-998c-16224795d2a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.469618 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.469580    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-892ds\" (UniqueName: \"kubernetes.io/projected/51a8af69-0b4c-4247-998c-16224795d2a2-kube-api-access-892ds\") pod \"cert-manager-operator-controller-manager-55c444bf86-xtf4d\" (UID: \"51a8af69-0b4c-4247-998c-16224795d2a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.469826 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.469671    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8af69-0b4c-4247-998c-16224795d2a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xtf4d\" (UID: \"51a8af69-0b4c-4247-998c-16224795d2a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.470016 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.469995    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8af69-0b4c-4247-998c-16224795d2a2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xtf4d\" (UID: \"51a8af69-0b4c-4247-998c-16224795d2a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.478215 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.478187    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-892ds\" (UniqueName: \"kubernetes.io/projected/51a8af69-0b4c-4247-998c-16224795d2a2-kube-api-access-892ds\") pod \"cert-manager-operator-controller-manager-55c444bf86-xtf4d\" (UID: \"51a8af69-0b4c-4247-998c-16224795d2a2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.617451 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.617363    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"
Jul 17 18:00:34.749509 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:34.749488    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d"]
Jul 17 18:00:34.752203 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:00:34.752175    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a8af69_0b4c_4247_998c_16224795d2a2.slice/crio-eac8362f5d0ceb4c946b473bb1ba1c58c63a0cc226862d168099be5a5be4f8d7 WatchSource:0}: Error finding container eac8362f5d0ceb4c946b473bb1ba1c58c63a0cc226862d168099be5a5be4f8d7: Status 404 returned error can't find the container with id eac8362f5d0ceb4c946b473bb1ba1c58c63a0cc226862d168099be5a5be4f8d7
Jul 17 18:00:35.071270 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:35.071238    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d" event={"ID":"51a8af69-0b4c-4247-998c-16224795d2a2","Type":"ContainerStarted","Data":"eac8362f5d0ceb4c946b473bb1ba1c58c63a0cc226862d168099be5a5be4f8d7"}
Jul 17 18:00:38.087375 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.087334    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d" event={"ID":"51a8af69-0b4c-4247-998c-16224795d2a2","Type":"ContainerStarted","Data":"e207eea23b591cf5e9c4350e5ff0099d0f3f2ba532acd176f9d73695af7d1b18"}
Jul 17 18:00:38.116119 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.116004    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xtf4d" podStartSLOduration=1.704123557 podStartE2EDuration="4.115987882s" totalImagesPullingTime="2.411864325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:38.115676055 +0000 UTC m=+235.701152304" watchObservedRunningTime="2026-07-17 18:00:38.115987882 +0000 UTC m=+235.701464134"
Jul 17 18:00:38.899312 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.899276    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"]
Jul 17 18:00:38.902778 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.902761    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:38.905252 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.905228    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 18:00:38.906248 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.906230    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 18:00:38.906360 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.906343    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m5qws\""
Jul 17 18:00:38.912224 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:38.912200    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"]
Jul 17 18:00:39.007233 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.007190    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml9r9\" (UniqueName: \"kubernetes.io/projected/a3fb87fd-c3be-4e55-882e-db91e6627e6a-kube-api-access-ml9r9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.007233 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.007239    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.007465 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.007276    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.107654 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.107605    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.108123 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.107673    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.108123 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.107728    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml9r9\" (UniqueName: \"kubernetes.io/projected/a3fb87fd-c3be-4e55-882e-db91e6627e6a-kube-api-access-ml9r9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.108123 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.108009    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.108123 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.108065    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.117987 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.117954    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml9r9\" (UniqueName: \"kubernetes.io/projected/a3fb87fd-c3be-4e55-882e-db91e6627e6a-kube-api-access-ml9r9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.212296 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.212201    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:39.343144 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:39.343121    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"]
Jul 17 18:00:39.345560 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:00:39.345519    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3fb87fd_c3be_4e55_882e_db91e6627e6a.slice/crio-d002c0886daf025895d8d659dde91e17f6eb43e50cbea00e3d78c3086b27766d WatchSource:0}: Error finding container d002c0886daf025895d8d659dde91e17f6eb43e50cbea00e3d78c3086b27766d: Status 404 returned error can't find the container with id d002c0886daf025895d8d659dde91e17f6eb43e50cbea00e3d78c3086b27766d
Jul 17 18:00:40.071671 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.071624    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"]
Jul 17 18:00:40.074844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.074825    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.077555 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.077535    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 18:00:40.077555 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.077546    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 18:00:40.078350 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.078333    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dtwnv\""
Jul 17 18:00:40.084065 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.084045    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"]
Jul 17 18:00:40.097032 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.097000    2555 generic.go:356] "Generic (PLEG): container finished" podID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerID="5e0236da51872a9f66c3239dbbdd45babe43293af422f10441462283c3099ea4" exitCode=0
Jul 17 18:00:40.097190 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.097046    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj" event={"ID":"a3fb87fd-c3be-4e55-882e-db91e6627e6a","Type":"ContainerDied","Data":"5e0236da51872a9f66c3239dbbdd45babe43293af422f10441462283c3099ea4"}
Jul 17 18:00:40.097190 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.097083    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj" event={"ID":"a3fb87fd-c3be-4e55-882e-db91e6627e6a","Type":"ContainerStarted","Data":"d002c0886daf025895d8d659dde91e17f6eb43e50cbea00e3d78c3086b27766d"}
Jul 17 18:00:40.116836 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.116807    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31e64fd1-ad74-4e91-a900-08d166f6350b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x6zpt\" (UID: \"31e64fd1-ad74-4e91-a900-08d166f6350b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.116836 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.116838    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5crj9\" (UniqueName: \"kubernetes.io/projected/31e64fd1-ad74-4e91-a900-08d166f6350b-kube-api-access-5crj9\") pod \"cert-manager-cainjector-559cdd8c87-x6zpt\" (UID: \"31e64fd1-ad74-4e91-a900-08d166f6350b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.218196 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.218159    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31e64fd1-ad74-4e91-a900-08d166f6350b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x6zpt\" (UID: \"31e64fd1-ad74-4e91-a900-08d166f6350b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.218196 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.218203    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5crj9\" (UniqueName: \"kubernetes.io/projected/31e64fd1-ad74-4e91-a900-08d166f6350b-kube-api-access-5crj9\") pod \"cert-manager-cainjector-559cdd8c87-x6zpt\" (UID: \"31e64fd1-ad74-4e91-a900-08d166f6350b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.226222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.226196    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31e64fd1-ad74-4e91-a900-08d166f6350b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x6zpt\" (UID: \"31e64fd1-ad74-4e91-a900-08d166f6350b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.226352 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.226205    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5crj9\" (UniqueName: \"kubernetes.io/projected/31e64fd1-ad74-4e91-a900-08d166f6350b-kube-api-access-5crj9\") pod \"cert-manager-cainjector-559cdd8c87-x6zpt\" (UID: \"31e64fd1-ad74-4e91-a900-08d166f6350b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.394144 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.394050    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"
Jul 17 18:00:40.522056 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:40.522023    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt"]
Jul 17 18:00:40.525032 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:00:40.525008    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31e64fd1_ad74_4e91_a900_08d166f6350b.slice/crio-8887023c5c35a1a2d135c7bb6aa84700b17fbbff528ad66ce9bb1522adf35c4e WatchSource:0}: Error finding container 8887023c5c35a1a2d135c7bb6aa84700b17fbbff528ad66ce9bb1522adf35c4e: Status 404 returned error can't find the container with id 8887023c5c35a1a2d135c7bb6aa84700b17fbbff528ad66ce9bb1522adf35c4e
Jul 17 18:00:41.102074 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:41.102043    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt" event={"ID":"31e64fd1-ad74-4e91-a900-08d166f6350b","Type":"ContainerStarted","Data":"8887023c5c35a1a2d135c7bb6aa84700b17fbbff528ad66ce9bb1522adf35c4e"}
Jul 17 18:00:43.113786 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:43.113696    2555 generic.go:356] "Generic (PLEG): container finished" podID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerID="6beaf51510cf8376feaa1d1e0dfcc0069c449b12b8cc8121337319818432de61" exitCode=0
Jul 17 18:00:43.113786 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:43.113759    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj" event={"ID":"a3fb87fd-c3be-4e55-882e-db91e6627e6a","Type":"ContainerDied","Data":"6beaf51510cf8376feaa1d1e0dfcc0069c449b12b8cc8121337319818432de61"}
Jul 17 18:00:44.120851 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:44.120816    2555 generic.go:356] "Generic (PLEG): container finished" podID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerID="b405eabe7b0e40bd2987217cb59e5051d0f83edb475f077f9dddb4616e43a157" exitCode=0
Jul 17 18:00:44.121278 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:44.120895    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj" event={"ID":"a3fb87fd-c3be-4e55-882e-db91e6627e6a","Type":"ContainerDied","Data":"b405eabe7b0e40bd2987217cb59e5051d0f83edb475f077f9dddb4616e43a157"}
Jul 17 18:00:45.126854 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.126815    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt" event={"ID":"31e64fd1-ad74-4e91-a900-08d166f6350b","Type":"ContainerStarted","Data":"834c4770f7aaf736e0be2c6361a6734b9c1b8323b82fcfe87d7a9893565bd50a"}
Jul 17 18:00:45.146215 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.146161    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x6zpt" podStartSLOduration=1.286341324 podStartE2EDuration="5.146142983s" totalImagesPullingTime="3.859801659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:45.145362549 +0000 UTC m=+242.730838828" watchObservedRunningTime="2026-07-17 18:00:45.146142983 +0000 UTC m=+242.731619237"
Jul 17 18:00:45.266441 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.266415    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:45.361500 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.361458    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml9r9\" (UniqueName: \"kubernetes.io/projected/a3fb87fd-c3be-4e55-882e-db91e6627e6a-kube-api-access-ml9r9\") pod \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") "
Jul 17 18:00:45.361500 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.361502    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-util\") pod \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") "
Jul 17 18:00:45.361736 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.361547    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-bundle\") pod \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\" (UID: \"a3fb87fd-c3be-4e55-882e-db91e6627e6a\") "
Jul 17 18:00:45.362033 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.362011    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-bundle" (OuterVolumeSpecName: "bundle") pod "a3fb87fd-c3be-4e55-882e-db91e6627e6a" (UID: "a3fb87fd-c3be-4e55-882e-db91e6627e6a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:00:45.363794 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.363768    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3fb87fd-c3be-4e55-882e-db91e6627e6a-kube-api-access-ml9r9" (OuterVolumeSpecName: "kube-api-access-ml9r9") pod "a3fb87fd-c3be-4e55-882e-db91e6627e6a" (UID: "a3fb87fd-c3be-4e55-882e-db91e6627e6a"). InnerVolumeSpecName "kube-api-access-ml9r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:00:45.365812 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.365788    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-util" (OuterVolumeSpecName: "util") pod "a3fb87fd-c3be-4e55-882e-db91e6627e6a" (UID: "a3fb87fd-c3be-4e55-882e-db91e6627e6a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:00:45.462949 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.462837    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml9r9\" (UniqueName: \"kubernetes.io/projected/a3fb87fd-c3be-4e55-882e-db91e6627e6a-kube-api-access-ml9r9\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:00:45.462949 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.462885    2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:00:45.462949 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.462900    2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3fb87fd-c3be-4e55-882e-db91e6627e6a-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:00:45.777016 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.776981    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ml2dk"]
Jul 17 18:00:45.777360 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.777341    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerName="pull"
Jul 17 18:00:45.777442 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.777362    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerName="pull"
Jul 17 18:00:45.777442 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.777392    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerName="extract"
Jul 17 18:00:45.777442 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.777400    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerName="extract"
Jul 17 18:00:45.777442 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.777414    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerName="util"
Jul 17 18:00:45.777442 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.777423    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerName="util"
Jul 17 18:00:45.777762 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.777497    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3fb87fd-c3be-4e55-882e-db91e6627e6a" containerName="extract"
Jul 17 18:00:45.780818 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.780796    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:45.783198 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.783173    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-q6zp7\""
Jul 17 18:00:45.790630 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.790608    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ml2dk"]
Jul 17 18:00:45.867543 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.867509    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d60544a-bd90-41c4-b6c0-beff284a5c5c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ml2dk\" (UID: \"4d60544a-bd90-41c4-b6c0-beff284a5c5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:45.867543 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.867549    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtlj5\" (UniqueName: \"kubernetes.io/projected/4d60544a-bd90-41c4-b6c0-beff284a5c5c-kube-api-access-dtlj5\") pod \"cert-manager-webhook-6486ffd997-ml2dk\" (UID: \"4d60544a-bd90-41c4-b6c0-beff284a5c5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:45.968843 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.968806    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d60544a-bd90-41c4-b6c0-beff284a5c5c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ml2dk\" (UID: \"4d60544a-bd90-41c4-b6c0-beff284a5c5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:45.968843 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.968857    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtlj5\" (UniqueName: \"kubernetes.io/projected/4d60544a-bd90-41c4-b6c0-beff284a5c5c-kube-api-access-dtlj5\") pod \"cert-manager-webhook-6486ffd997-ml2dk\" (UID: \"4d60544a-bd90-41c4-b6c0-beff284a5c5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:45.977160 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.977115    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d60544a-bd90-41c4-b6c0-beff284a5c5c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ml2dk\" (UID: \"4d60544a-bd90-41c4-b6c0-beff284a5c5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:45.977299 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:45.977231    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtlj5\" (UniqueName: \"kubernetes.io/projected/4d60544a-bd90-41c4-b6c0-beff284a5c5c-kube-api-access-dtlj5\") pod \"cert-manager-webhook-6486ffd997-ml2dk\" (UID: \"4d60544a-bd90-41c4-b6c0-beff284a5c5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:46.089724 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:46.089593    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:46.134210 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:46.134167    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj" event={"ID":"a3fb87fd-c3be-4e55-882e-db91e6627e6a","Type":"ContainerDied","Data":"d002c0886daf025895d8d659dde91e17f6eb43e50cbea00e3d78c3086b27766d"}
Jul 17 18:00:46.134668 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:46.134197    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff9cdj"
Jul 17 18:00:46.134668 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:46.134233    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d002c0886daf025895d8d659dde91e17f6eb43e50cbea00e3d78c3086b27766d"
Jul 17 18:00:46.251926 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:46.251894    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ml2dk"]
Jul 17 18:00:46.255058 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:00:46.255026    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d60544a_bd90_41c4_b6c0_beff284a5c5c.slice/crio-3f67d659b9f561b437723fdeadaf0e67748c32f9e9cafd42bef0f827ffcd0550 WatchSource:0}: Error finding container 3f67d659b9f561b437723fdeadaf0e67748c32f9e9cafd42bef0f827ffcd0550: Status 404 returned error can't find the container with id 3f67d659b9f561b437723fdeadaf0e67748c32f9e9cafd42bef0f827ffcd0550
Jul 17 18:00:47.139786 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:47.139751    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk" event={"ID":"4d60544a-bd90-41c4-b6c0-beff284a5c5c","Type":"ContainerStarted","Data":"6afd9a7c9bed1b1a30473978838074b6172b208372bf8862de459bb2cede8f48"}
Jul 17 18:00:47.139786 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:47.139790    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk" event={"ID":"4d60544a-bd90-41c4-b6c0-beff284a5c5c","Type":"ContainerStarted","Data":"3f67d659b9f561b437723fdeadaf0e67748c32f9e9cafd42bef0f827ffcd0550"}
Jul 17 18:00:47.140214 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:47.139866    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:47.190476 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:47.190425    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk" podStartSLOduration=2.19040808 podStartE2EDuration="2.19040808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:47.189999446 +0000 UTC m=+244.775475699" watchObservedRunningTime="2026-07-17 18:00:47.19040808 +0000 UTC m=+244.775884331"
Jul 17 18:00:49.852394 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:49.852358    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mgxnk"]
Jul 17 18:00:49.859772 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:49.859745    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:49.862407 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:49.862383    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qr2dl\""
Jul 17 18:00:49.865846 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:49.865818    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mgxnk"]
Jul 17 18:00:50.003313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.003268    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7rgb\" (UniqueName: \"kubernetes.io/projected/6f414e1d-fb6d-44db-84a9-3584df34edd3-kube-api-access-w7rgb\") pod \"cert-manager-55b4d954d5-mgxnk\" (UID: \"6f414e1d-fb6d-44db-84a9-3584df34edd3\") " pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:50.003313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.003311    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f414e1d-fb6d-44db-84a9-3584df34edd3-bound-sa-token\") pod \"cert-manager-55b4d954d5-mgxnk\" (UID: \"6f414e1d-fb6d-44db-84a9-3584df34edd3\") " pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:50.104560 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.104470    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7rgb\" (UniqueName: \"kubernetes.io/projected/6f414e1d-fb6d-44db-84a9-3584df34edd3-kube-api-access-w7rgb\") pod \"cert-manager-55b4d954d5-mgxnk\" (UID: \"6f414e1d-fb6d-44db-84a9-3584df34edd3\") " pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:50.104560 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.104507    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f414e1d-fb6d-44db-84a9-3584df34edd3-bound-sa-token\") pod \"cert-manager-55b4d954d5-mgxnk\" (UID: \"6f414e1d-fb6d-44db-84a9-3584df34edd3\") " pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:50.114233 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.114196    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f414e1d-fb6d-44db-84a9-3584df34edd3-bound-sa-token\") pod \"cert-manager-55b4d954d5-mgxnk\" (UID: \"6f414e1d-fb6d-44db-84a9-3584df34edd3\") " pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:50.114381 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.114311    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7rgb\" (UniqueName: \"kubernetes.io/projected/6f414e1d-fb6d-44db-84a9-3584df34edd3-kube-api-access-w7rgb\") pod \"cert-manager-55b4d954d5-mgxnk\" (UID: \"6f414e1d-fb6d-44db-84a9-3584df34edd3\") " pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:50.170147 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.170105    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mgxnk"
Jul 17 18:00:50.307257 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:50.307227    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mgxnk"]
Jul 17 18:00:50.308212 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:00:50.308183    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f414e1d_fb6d_44db_84a9_3584df34edd3.slice/crio-97d42a45062a20e68ba1c2ef7105d03bedb1f13256705fcb754726375b64ac59 WatchSource:0}: Error finding container 97d42a45062a20e68ba1c2ef7105d03bedb1f13256705fcb754726375b64ac59: Status 404 returned error can't find the container with id 97d42a45062a20e68ba1c2ef7105d03bedb1f13256705fcb754726375b64ac59
Jul 17 18:00:51.158313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:51.158283    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mgxnk" event={"ID":"6f414e1d-fb6d-44db-84a9-3584df34edd3","Type":"ContainerStarted","Data":"181de68d6784e0b09dc5f20385c1e16a68d3180ab97d635ab555a74bea1adc2b"}
Jul 17 18:00:51.158313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:51.158320    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mgxnk" event={"ID":"6f414e1d-fb6d-44db-84a9-3584df34edd3","Type":"ContainerStarted","Data":"97d42a45062a20e68ba1c2ef7105d03bedb1f13256705fcb754726375b64ac59"}
Jul 17 18:00:51.179353 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:51.179295    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mgxnk" podStartSLOduration=2.179274869 podStartE2EDuration="2.179274869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:00:51.178377371 +0000 UTC m=+248.763853626" watchObservedRunningTime="2026-07-17 18:00:51.179274869 +0000 UTC m=+248.764751122"
Jul 17 18:00:53.146981 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:53.146953    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ml2dk"
Jul 17 18:00:58.796999 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.796964    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"]
Jul 17 18:00:58.799340 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.799323    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.803799 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.803774    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 18:00:58.804002 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.803984    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m5qws\""
Jul 17 18:00:58.804680 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.804664    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 18:00:58.813966 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.813935    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"]
Jul 17 18:00:58.886304 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.886268    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.886451 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.886314    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl7bn\" (UniqueName: \"kubernetes.io/projected/e7859799-6574-487d-b987-280b22695f88-kube-api-access-nl7bn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.886451 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.886336    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.987565 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.987525    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.987565 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.987573    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl7bn\" (UniqueName: \"kubernetes.io/projected/e7859799-6574-487d-b987-280b22695f88-kube-api-access-nl7bn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.987817 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.987707    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.988010 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.987991    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.988082 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.988027    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:58.997874 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:58.997843    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl7bn\" (UniqueName: \"kubernetes.io/projected/e7859799-6574-487d-b987-280b22695f88-kube-api-access-nl7bn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:59.109463 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:59.109365    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:00:59.247785 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:00:59.247747    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"]
Jul 17 18:00:59.251326 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:00:59.251295    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7859799_6574_487d_b987_280b22695f88.slice/crio-28160a5c8911087b14ce3f94a6584ffdee2c7e7b2ebe0c238357b091c6227b05 WatchSource:0}: Error finding container 28160a5c8911087b14ce3f94a6584ffdee2c7e7b2ebe0c238357b091c6227b05: Status 404 returned error can't find the container with id 28160a5c8911087b14ce3f94a6584ffdee2c7e7b2ebe0c238357b091c6227b05
Jul 17 18:01:00.201074 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:00.201042    2555 generic.go:356] "Generic (PLEG): container finished" podID="e7859799-6574-487d-b987-280b22695f88" containerID="25ea4483b33760b1afe8524577950c659fe7308c47f5a4fd3c649d1c7df69649" exitCode=0
Jul 17 18:01:00.201490 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:00.201106    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76" event={"ID":"e7859799-6574-487d-b987-280b22695f88","Type":"ContainerDied","Data":"25ea4483b33760b1afe8524577950c659fe7308c47f5a4fd3c649d1c7df69649"}
Jul 17 18:01:00.201490 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:00.201146    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76" event={"ID":"e7859799-6574-487d-b987-280b22695f88","Type":"ContainerStarted","Data":"28160a5c8911087b14ce3f94a6584ffdee2c7e7b2ebe0c238357b091c6227b05"}
Jul 17 18:01:01.208301 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:01.208217    2555 generic.go:356] "Generic (PLEG): container finished" podID="e7859799-6574-487d-b987-280b22695f88" containerID="13f6e12b12dc1eec91d74fee9974d3c70667ae17013eb115a936f49d069d25ba" exitCode=0
Jul 17 18:01:01.208301 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:01.208274    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76" event={"ID":"e7859799-6574-487d-b987-280b22695f88","Type":"ContainerDied","Data":"13f6e12b12dc1eec91d74fee9974d3c70667ae17013eb115a936f49d069d25ba"}
Jul 17 18:01:02.215589 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:02.215549    2555 generic.go:356] "Generic (PLEG): container finished" podID="e7859799-6574-487d-b987-280b22695f88" containerID="410675ecd0d67aafd84aefb7cd5a58e0a82596c1989d3cdfb7b9755f8710c2a5" exitCode=0
Jul 17 18:01:02.216078 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:02.215590    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76" event={"ID":"e7859799-6574-487d-b987-280b22695f88","Type":"ContainerDied","Data":"410675ecd0d67aafd84aefb7cd5a58e0a82596c1989d3cdfb7b9755f8710c2a5"}
Jul 17 18:01:03.348386 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.348359    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:01:03.526862 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.526827    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-util\") pod \"e7859799-6574-487d-b987-280b22695f88\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") "
Jul 17 18:01:03.527063 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.526888    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-bundle\") pod \"e7859799-6574-487d-b987-280b22695f88\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") "
Jul 17 18:01:03.527063 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.526920    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl7bn\" (UniqueName: \"kubernetes.io/projected/e7859799-6574-487d-b987-280b22695f88-kube-api-access-nl7bn\") pod \"e7859799-6574-487d-b987-280b22695f88\" (UID: \"e7859799-6574-487d-b987-280b22695f88\") "
Jul 17 18:01:03.527818 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.527784    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-bundle" (OuterVolumeSpecName: "bundle") pod "e7859799-6574-487d-b987-280b22695f88" (UID: "e7859799-6574-487d-b987-280b22695f88"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:01:03.529156 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.529134    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7859799-6574-487d-b987-280b22695f88-kube-api-access-nl7bn" (OuterVolumeSpecName: "kube-api-access-nl7bn") pod "e7859799-6574-487d-b987-280b22695f88" (UID: "e7859799-6574-487d-b987-280b22695f88"). InnerVolumeSpecName "kube-api-access-nl7bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:01:03.533396 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.533269    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-util" (OuterVolumeSpecName: "util") pod "e7859799-6574-487d-b987-280b22695f88" (UID: "e7859799-6574-487d-b987-280b22695f88"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:01:03.627783 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.627743    2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:03.627783 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.627776    2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7859799-6574-487d-b987-280b22695f88-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:03.627783 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:03.627789    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl7bn\" (UniqueName: \"kubernetes.io/projected/e7859799-6574-487d-b987-280b22695f88-kube-api-access-nl7bn\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:04.227316 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:04.227282    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76" event={"ID":"e7859799-6574-487d-b987-280b22695f88","Type":"ContainerDied","Data":"28160a5c8911087b14ce3f94a6584ffdee2c7e7b2ebe0c238357b091c6227b05"}
Jul 17 18:01:04.227497 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:04.227331    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28160a5c8911087b14ce3f94a6584ffdee2c7e7b2ebe0c238357b091c6227b05"
Jul 17 18:01:04.227497 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:04.227299    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2xt76"
Jul 17 18:01:12.050249 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050216    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"]
Jul 17 18:01:12.050702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050546    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7859799-6574-487d-b987-280b22695f88" containerName="extract"
Jul 17 18:01:12.050702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050556    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7859799-6574-487d-b987-280b22695f88" containerName="extract"
Jul 17 18:01:12.050702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050567    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7859799-6574-487d-b987-280b22695f88" containerName="util"
Jul 17 18:01:12.050702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050573    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7859799-6574-487d-b987-280b22695f88" containerName="util"
Jul 17 18:01:12.050702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050582    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7859799-6574-487d-b987-280b22695f88" containerName="pull"
Jul 17 18:01:12.050702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050588    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7859799-6574-487d-b987-280b22695f88" containerName="pull"
Jul 17 18:01:12.050702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.050652    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7859799-6574-487d-b987-280b22695f88" containerName="extract"
Jul 17 18:01:12.056057 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.056038    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.060278 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.060255    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 18:01:12.060425 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.060358    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 18:01:12.060425 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.060365    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p5jsc\""
Jul 17 18:01:12.060425 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.060403    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 18:01:12.060425 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.060410    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 18:01:12.070847 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.070819    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"]
Jul 17 18:01:12.101656 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.101621    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/047e5ea0-35dd-4779-b821-27e1234abbcc-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.101831 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.101683    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv782\" (UniqueName: \"kubernetes.io/projected/047e5ea0-35dd-4779-b821-27e1234abbcc-kube-api-access-nv782\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.101831 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.101711    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/047e5ea0-35dd-4779-b821-27e1234abbcc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.101831 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.101739    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/047e5ea0-35dd-4779-b821-27e1234abbcc-tmp\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.101831 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.101791    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/047e5ea0-35dd-4779-b821-27e1234abbcc-manifests\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.202377 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.202334    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/047e5ea0-35dd-4779-b821-27e1234abbcc-tmp\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.202377 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.202385    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/047e5ea0-35dd-4779-b821-27e1234abbcc-manifests\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.202683 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.202425    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/047e5ea0-35dd-4779-b821-27e1234abbcc-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.202683 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.202443    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv782\" (UniqueName: \"kubernetes.io/projected/047e5ea0-35dd-4779-b821-27e1234abbcc-kube-api-access-nv782\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.202683 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.202470    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/047e5ea0-35dd-4779-b821-27e1234abbcc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.202868 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.202840    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/047e5ea0-35dd-4779-b821-27e1234abbcc-manifests\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.204761 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.204728    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/047e5ea0-35dd-4779-b821-27e1234abbcc-tmp\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.204963 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.204942    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/047e5ea0-35dd-4779-b821-27e1234abbcc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.205005 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.204987    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/047e5ea0-35dd-4779-b821-27e1234abbcc-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.215003 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.214974    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv782\" (UniqueName: \"kubernetes.io/projected/047e5ea0-35dd-4779-b821-27e1234abbcc-kube-api-access-nv782\") pod \"opendatahub-operator-controller-manager-5d8bdcf84-72fkb\" (UID: \"047e5ea0-35dd-4779-b821-27e1234abbcc\") " pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.372629 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.372541    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:12.515864 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:12.515838    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"]
Jul 17 18:01:12.518663 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:01:12.518620    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047e5ea0_35dd_4779_b821_27e1234abbcc.slice/crio-aaa6384c91135420561abc829091d51989aca7072ef53cd54500cc14aff52283 WatchSource:0}: Error finding container aaa6384c91135420561abc829091d51989aca7072ef53cd54500cc14aff52283: Status 404 returned error can't find the container with id aaa6384c91135420561abc829091d51989aca7072ef53cd54500cc14aff52283
Jul 17 18:01:13.270255 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:13.270157    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb" event={"ID":"047e5ea0-35dd-4779-b821-27e1234abbcc","Type":"ContainerStarted","Data":"aaa6384c91135420561abc829091d51989aca7072ef53cd54500cc14aff52283"}
Jul 17 18:01:16.286206 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:16.286174    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb" event={"ID":"047e5ea0-35dd-4779-b821-27e1234abbcc","Type":"ContainerStarted","Data":"e4140b73ebed4cd5835ab9df9d7c829a84cd02f7e1151faea83c9629eee62917"}
Jul 17 18:01:17.292338 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:17.292302    2555 generic.go:356] "Generic (PLEG): container finished" podID="047e5ea0-35dd-4779-b821-27e1234abbcc" containerID="e4140b73ebed4cd5835ab9df9d7c829a84cd02f7e1151faea83c9629eee62917" exitCode=0
Jul 17 18:01:17.292747 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:17.292390    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb" event={"ID":"047e5ea0-35dd-4779-b821-27e1234abbcc","Type":"ContainerDied","Data":"e4140b73ebed4cd5835ab9df9d7c829a84cd02f7e1151faea83c9629eee62917"}
Jul 17 18:01:19.303157 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:19.303124    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb" event={"ID":"047e5ea0-35dd-4779-b821-27e1234abbcc","Type":"ContainerStarted","Data":"c8d0b7a829890806c2874efb311ebb39b108c21606e8a1ac170090714aaa56ab"}
Jul 17 18:01:19.303607 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:19.303260    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:19.324502 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:19.324449    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb" podStartSLOduration=1.928326988 podStartE2EDuration="7.32443149s" totalImagesPullingTime="4.311244694s" totalInitContainerRuntime="1.084859808s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:19.322554198 +0000 UTC m=+276.908030467" watchObservedRunningTime="2026-07-17 18:01:19.32443149 +0000 UTC m=+276.909907818"
Jul 17 18:01:30.310038 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:30.310007    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d8bdcf84-72fkb"
Jul 17 18:01:32.477624 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.477588    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"]
Jul 17 18:01:32.485867 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.485838    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.489149 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.489125    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 18:01:32.489848 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.489829    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 18:01:32.490363 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.490348    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 18:01:32.491261 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.491234    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 18:01:32.491261 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.491252    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r5k6d\""
Jul 17 18:01:32.510442 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.510414    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"]
Jul 17 18:01:32.563566 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.563536    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa1a105c-005e-47b6-8093-036767bd1b35-tmp\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.563769 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.563596    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28bwp\" (UniqueName: \"kubernetes.io/projected/aa1a105c-005e-47b6-8093-036767bd1b35-kube-api-access-28bwp\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.563769 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.563685    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1a105c-005e-47b6-8093-036767bd1b35-tls-certs\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.664584 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.664542    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dr8fl"]
Jul 17 18:01:32.664867 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.664845    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa1a105c-005e-47b6-8093-036767bd1b35-tmp\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.664916 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.664900    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28bwp\" (UniqueName: \"kubernetes.io/projected/aa1a105c-005e-47b6-8093-036767bd1b35-kube-api-access-28bwp\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.665050 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.665030    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1a105c-005e-47b6-8093-036767bd1b35-tls-certs\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.665168 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:32.665152    2555 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 17 18:01:32.665241 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:32.665223    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa1a105c-005e-47b6-8093-036767bd1b35-tls-certs podName:aa1a105c-005e-47b6-8093-036767bd1b35 nodeName:}" failed. No retries permitted until 2026-07-17 18:01:33.165205102 +0000 UTC m=+290.750681344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/aa1a105c-005e-47b6-8093-036767bd1b35-tls-certs") pod "kube-auth-proxy-84bfff449b-qmvlc" (UID: "aa1a105c-005e-47b6-8093-036767bd1b35") : secret "kube-auth-proxy-tls" not found
Jul 17 18:01:32.667405 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.667368    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa1a105c-005e-47b6-8093-036767bd1b35-tmp\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.668247 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.668229    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:32.675226 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.675210    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 17 18:01:32.677001 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.676980    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-q6trz\""
Jul 17 18:01:32.683192 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.683139    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dr8fl"]
Jul 17 18:01:32.694022 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.693991    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28bwp\" (UniqueName: \"kubernetes.io/projected/aa1a105c-005e-47b6-8093-036767bd1b35-kube-api-access-28bwp\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:32.766326 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.766273    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svhmt\" (UniqueName: \"kubernetes.io/projected/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-kube-api-access-svhmt\") pod \"odh-model-controller-7998c67dc4-dr8fl\" (UID: \"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:32.766326 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.766338    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-cert\") pod \"odh-model-controller-7998c67dc4-dr8fl\" (UID: \"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:32.867398 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.867360    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svhmt\" (UniqueName: \"kubernetes.io/projected/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-kube-api-access-svhmt\") pod \"odh-model-controller-7998c67dc4-dr8fl\" (UID: \"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:32.867398 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.867407    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-cert\") pod \"odh-model-controller-7998c67dc4-dr8fl\" (UID: \"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:32.867743 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:32.867567    2555 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 17 18:01:32.867743 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:32.867673    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-cert podName:1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4 nodeName:}" failed. No retries permitted until 2026-07-17 18:01:33.367624221 +0000 UTC m=+290.953100474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-cert") pod "odh-model-controller-7998c67dc4-dr8fl" (UID: "1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4") : secret "odh-model-controller-webhook-cert" not found
Jul 17 18:01:32.885607 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:32.885573    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svhmt\" (UniqueName: \"kubernetes.io/projected/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-kube-api-access-svhmt\") pod \"odh-model-controller-7998c67dc4-dr8fl\" (UID: \"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:33.170411 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.170318    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1a105c-005e-47b6-8093-036767bd1b35-tls-certs\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:33.173176 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.173148    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1a105c-005e-47b6-8093-036767bd1b35-tls-certs\") pod \"kube-auth-proxy-84bfff449b-qmvlc\" (UID: \"aa1a105c-005e-47b6-8093-036767bd1b35\") " pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:33.371590 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.371555    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-cert\") pod \"odh-model-controller-7998c67dc4-dr8fl\" (UID: \"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:33.373975 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.373953    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4-cert\") pod \"odh-model-controller-7998c67dc4-dr8fl\" (UID: \"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:33.395838 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.395803    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"
Jul 17 18:01:33.548224 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.548080    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc"]
Jul 17 18:01:33.550657 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:01:33.550618    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa1a105c_005e_47b6_8093_036767bd1b35.slice/crio-c19d65a1466a5fe6d65be1efe49febd8f2cacb5d2ff83ca9d0c95414aabcf739 WatchSource:0}: Error finding container c19d65a1466a5fe6d65be1efe49febd8f2cacb5d2ff83ca9d0c95414aabcf739: Status 404 returned error can't find the container with id c19d65a1466a5fe6d65be1efe49febd8f2cacb5d2ff83ca9d0c95414aabcf739
Jul 17 18:01:33.585740 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.585709    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:33.731572 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.727863    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dr8fl"]
Jul 17 18:01:33.748156 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.748127    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qwmw7"]
Jul 17 18:01:33.751589 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.751571    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:33.753814 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.753797    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 17 18:01:33.753889 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.753849    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-lbl4b\""
Jul 17 18:01:33.764744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.764721    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qwmw7"]
Jul 17 18:01:33.876548 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.876513    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02eb8a4d-4c18-4663-88ec-a19d0fab22a7-tls-certs\") pod \"model-serving-api-54d89bd5fc-qwmw7\" (UID: \"02eb8a4d-4c18-4663-88ec-a19d0fab22a7\") " pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:33.876548 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.876553    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz78d\" (UniqueName: \"kubernetes.io/projected/02eb8a4d-4c18-4663-88ec-a19d0fab22a7-kube-api-access-xz78d\") pod \"model-serving-api-54d89bd5fc-qwmw7\" (UID: \"02eb8a4d-4c18-4663-88ec-a19d0fab22a7\") " pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:33.969803 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.969764    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"]
Jul 17 18:01:33.972990 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.972967    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:33.976075 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.976049    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 18:01:33.976259 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.976244    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 18:01:33.976344 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.976244    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 18:01:33.976344 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.976246    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 18:01:33.977056 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.977040    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02eb8a4d-4c18-4663-88ec-a19d0fab22a7-tls-certs\") pod \"model-serving-api-54d89bd5fc-qwmw7\" (UID: \"02eb8a4d-4c18-4663-88ec-a19d0fab22a7\") " pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:33.977105 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.977067    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz78d\" (UniqueName: \"kubernetes.io/projected/02eb8a4d-4c18-4663-88ec-a19d0fab22a7-kube-api-access-xz78d\") pod \"model-serving-api-54d89bd5fc-qwmw7\" (UID: \"02eb8a4d-4c18-4663-88ec-a19d0fab22a7\") " pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:33.977414 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.977391    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h2ctv\""
Jul 17 18:01:33.979735 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.979710    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02eb8a4d-4c18-4663-88ec-a19d0fab22a7-tls-certs\") pod \"model-serving-api-54d89bd5fc-qwmw7\" (UID: \"02eb8a4d-4c18-4663-88ec-a19d0fab22a7\") " pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:33.988429 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:33.988368    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz78d\" (UniqueName: \"kubernetes.io/projected/02eb8a4d-4c18-4663-88ec-a19d0fab22a7-kube-api-access-xz78d\") pod \"model-serving-api-54d89bd5fc-qwmw7\" (UID: \"02eb8a4d-4c18-4663-88ec-a19d0fab22a7\") " pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:34.004131 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.004101    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"]
Jul 17 18:01:34.062023 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.061984    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:34.078226 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.078197    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt2dh\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-kube-api-access-pt2dh\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.078353 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.078251    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.078353 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.078302    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e307756f-ae3d-429a-b757-8edd7793a97d-local-certs\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.078353 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.078331    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-istio-token\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.078467 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.078357    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-cacerts\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.078467 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.078373    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.078467 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.078396    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.179440 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.179404    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e307756f-ae3d-429a-b757-8edd7793a97d-local-certs\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.179627 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.179466    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-istio-token\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.179627 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.179507    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-cacerts\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.179627 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.179529    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.179627 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.179557    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.179627 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.179623    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt2dh\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-kube-api-access-pt2dh\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.179970 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.179702    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.182307 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.182265    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.182812 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.182753    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-cacerts\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.183001 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.182982    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.183123 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.183097    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e307756f-ae3d-429a-b757-8edd7793a97d-local-certs\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.183523 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.183504    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.208211 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.208185    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qwmw7"]
Jul 17 18:01:34.210512 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:01:34.210485    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02eb8a4d_4c18_4663_88ec_a19d0fab22a7.slice/crio-f25e254f974858fd3494896d400aedaef1d29f6a229cad63461297d99ed29509 WatchSource:0}: Error finding container f25e254f974858fd3494896d400aedaef1d29f6a229cad63461297d99ed29509: Status 404 returned error can't find the container with id f25e254f974858fd3494896d400aedaef1d29f6a229cad63461297d99ed29509
Jul 17 18:01:34.211120 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.211094    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt2dh\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-kube-api-access-pt2dh\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.211366 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.211347    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-istio-token\") pod \"istiod-openshift-gateway-7574459656-j2sdc\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.288981 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.288944    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:34.374979 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.374944    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl" event={"ID":"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4","Type":"ContainerStarted","Data":"3deb4146d944544b148354e49b05ae31a362ab3aa3b22bed664318f9f978a37a"}
Jul 17 18:01:34.376436 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.376397    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc" event={"ID":"aa1a105c-005e-47b6-8093-036767bd1b35","Type":"ContainerStarted","Data":"c19d65a1466a5fe6d65be1efe49febd8f2cacb5d2ff83ca9d0c95414aabcf739"}
Jul 17 18:01:34.378157 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.378130    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7" event={"ID":"02eb8a4d-4c18-4663-88ec-a19d0fab22a7","Type":"ContainerStarted","Data":"f25e254f974858fd3494896d400aedaef1d29f6a229cad63461297d99ed29509"}
Jul 17 18:01:34.452347 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:34.452280    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"]
Jul 17 18:01:34.454486 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:01:34.454460    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode307756f_ae3d_429a_b757_8edd7793a97d.slice/crio-c5e1a446dc7c55249f2a874b84cd7a738e776f319cf1d4445fa742d376360975 WatchSource:0}: Error finding container c5e1a446dc7c55249f2a874b84cd7a738e776f319cf1d4445fa742d376360975: Status 404 returned error can't find the container with id c5e1a446dc7c55249f2a874b84cd7a738e776f319cf1d4445fa742d376360975
Jul 17 18:01:35.385416 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:35.385373    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc" event={"ID":"e307756f-ae3d-429a-b757-8edd7793a97d","Type":"ContainerStarted","Data":"c5e1a446dc7c55249f2a874b84cd7a738e776f319cf1d4445fa742d376360975"}
Jul 17 18:01:39.237271 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.237220    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 18:01:39.237676 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.237277    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 18:01:39.410187 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.410147    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl" event={"ID":"1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4","Type":"ContainerStarted","Data":"eae83693b7cbb28f6ed77a9b054dd622066cbfc895ba8149640f6eeac2d15d02"}
Jul 17 18:01:39.410364 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.410270    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:01:39.411734 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.411705    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc" event={"ID":"aa1a105c-005e-47b6-8093-036767bd1b35","Type":"ContainerStarted","Data":"c0983e75598b766bce5f63474deefe0692f5d7e396ba4cf8f996e0a914952d97"}
Jul 17 18:01:39.413418 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.413393    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc" event={"ID":"e307756f-ae3d-429a-b757-8edd7793a97d","Type":"ContainerStarted","Data":"86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6"}
Jul 17 18:01:39.413514 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.413443    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:39.415139 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.415118    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7" event={"ID":"02eb8a4d-4c18-4663-88ec-a19d0fab22a7","Type":"ContainerStarted","Data":"e8aa129a9f0408515dca75ccd95ee7933eccb4194b51beb62e3c17c6e04f2372"}
Jul 17 18:01:39.415248 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.415169    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:39.430460 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.430398    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl" podStartSLOduration=1.919865546 podStartE2EDuration="7.430379361s" totalImagesPullingTime="5.510513815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:39.429940157 +0000 UTC m=+297.015416409" watchObservedRunningTime="2026-07-17 18:01:39.430379361 +0000 UTC m=+297.015855615"
Jul 17 18:01:39.449034 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.448981    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7" podStartSLOduration=1.430333719 podStartE2EDuration="6.44896003s" totalImagesPullingTime="5.018626311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:39.448224885 +0000 UTC m=+297.033701154" watchObservedRunningTime="2026-07-17 18:01:39.44896003 +0000 UTC m=+297.034436276"
Jul 17 18:01:39.467700 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.467582    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84bfff449b-qmvlc" podStartSLOduration=1.790463694 podStartE2EDuration="7.467564795s" totalImagesPullingTime="5.677101101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:39.466513694 +0000 UTC m=+297.051989943" watchObservedRunningTime="2026-07-17 18:01:39.467564795 +0000 UTC m=+297.053041045"
Jul 17 18:01:39.493281 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:39.493232    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc" podStartSLOduration=1.713228355 podStartE2EDuration="6.493214462s" totalImagesPullingTime="4.779986107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:39.490878657 +0000 UTC m=+297.076354910" watchObservedRunningTime="2026-07-17 18:01:39.493214462 +0000 UTC m=+297.078690715"
Jul 17 18:01:40.424731 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:40.424690    2555 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-j2sdc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 18:01:40.425162 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:40.424751    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc" podUID="e307756f-ae3d-429a-b757-8edd7793a97d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 18:01:42.862482 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:42.862451    2555 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 18:01:43.423405 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.423376    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:43.582654 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.582596    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"]
Jul 17 18:01:43.586218 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.586193    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"]
Jul 17 18:01:43.586377 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.586310    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:43.588855 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.588832    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2n8dt\""
Jul 17 18:01:43.588969 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.588832    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 18:01:43.589495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.589480    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:43.591248 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.591224    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 18:01:43.591870 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.591853    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-xp7g5\""
Jul 17 18:01:43.608795 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.608768    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"]
Jul 17 18:01:43.623605 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.623568    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"]
Jul 17 18:01:43.673922 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.673826    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e7f6181-4f4b-4ef7-a269-b847f43ba563-cert\") pod \"kserve-controller-manager-d6f4b65b8-fb5cz\" (UID: \"1e7f6181-4f4b-4ef7-a269-b847f43ba563\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:43.673922 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.673912    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cpxwg\" (UID: \"2cb9e2ac-f565-44c3-a832-4a81a349cc4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:43.674114 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.673963    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r45rp\" (UniqueName: \"kubernetes.io/projected/1e7f6181-4f4b-4ef7-a269-b847f43ba563-kube-api-access-r45rp\") pod \"kserve-controller-manager-d6f4b65b8-fb5cz\" (UID: \"1e7f6181-4f4b-4ef7-a269-b847f43ba563\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:43.674114 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.673989    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfp8j\" (UniqueName: \"kubernetes.io/projected/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-kube-api-access-zfp8j\") pod \"llmisvc-controller-manager-777c6bd7d5-cpxwg\" (UID: \"2cb9e2ac-f565-44c3-a832-4a81a349cc4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:43.774442 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.774405    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e7f6181-4f4b-4ef7-a269-b847f43ba563-cert\") pod \"kserve-controller-manager-d6f4b65b8-fb5cz\" (UID: \"1e7f6181-4f4b-4ef7-a269-b847f43ba563\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:43.774619 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.774461    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cpxwg\" (UID: \"2cb9e2ac-f565-44c3-a832-4a81a349cc4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:43.774619 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.774504    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r45rp\" (UniqueName: \"kubernetes.io/projected/1e7f6181-4f4b-4ef7-a269-b847f43ba563-kube-api-access-r45rp\") pod \"kserve-controller-manager-d6f4b65b8-fb5cz\" (UID: \"1e7f6181-4f4b-4ef7-a269-b847f43ba563\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:43.774619 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.774522    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfp8j\" (UniqueName: \"kubernetes.io/projected/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-kube-api-access-zfp8j\") pod \"llmisvc-controller-manager-777c6bd7d5-cpxwg\" (UID: \"2cb9e2ac-f565-44c3-a832-4a81a349cc4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:43.774619 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:43.774584    2555 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 17 18:01:43.774619 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:43.774608    2555 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 17 18:01:43.774817 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:43.774689    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-cert podName:2cb9e2ac-f565-44c3-a832-4a81a349cc4c nodeName:}" failed. No retries permitted until 2026-07-17 18:01:44.274664403 +0000 UTC m=+301.860140656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-cert") pod "llmisvc-controller-manager-777c6bd7d5-cpxwg" (UID: "2cb9e2ac-f565-44c3-a832-4a81a349cc4c") : secret "llmisvc-webhook-server-cert" not found
Jul 17 18:01:43.774817 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:43.774704    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e7f6181-4f4b-4ef7-a269-b847f43ba563-cert podName:1e7f6181-4f4b-4ef7-a269-b847f43ba563 nodeName:}" failed. No retries permitted until 2026-07-17 18:01:44.274698153 +0000 UTC m=+301.860174383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e7f6181-4f4b-4ef7-a269-b847f43ba563-cert") pod "kserve-controller-manager-d6f4b65b8-fb5cz" (UID: "1e7f6181-4f4b-4ef7-a269-b847f43ba563") : secret "kserve-webhook-server-cert" not found
Jul 17 18:01:43.784102 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.784072    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfp8j\" (UniqueName: \"kubernetes.io/projected/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-kube-api-access-zfp8j\") pod \"llmisvc-controller-manager-777c6bd7d5-cpxwg\" (UID: \"2cb9e2ac-f565-44c3-a832-4a81a349cc4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:43.784242 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:43.784077    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r45rp\" (UniqueName: \"kubernetes.io/projected/1e7f6181-4f4b-4ef7-a269-b847f43ba563-kube-api-access-r45rp\") pod \"kserve-controller-manager-d6f4b65b8-fb5cz\" (UID: \"1e7f6181-4f4b-4ef7-a269-b847f43ba563\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:44.279034 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.279002    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cpxwg\" (UID: \"2cb9e2ac-f565-44c3-a832-4a81a349cc4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:44.279466 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.279076    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e7f6181-4f4b-4ef7-a269-b847f43ba563-cert\") pod \"kserve-controller-manager-d6f4b65b8-fb5cz\" (UID: \"1e7f6181-4f4b-4ef7-a269-b847f43ba563\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:44.281556 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.281525    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cb9e2ac-f565-44c3-a832-4a81a349cc4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cpxwg\" (UID: \"2cb9e2ac-f565-44c3-a832-4a81a349cc4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:44.281686 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.281525    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e7f6181-4f4b-4ef7-a269-b847f43ba563-cert\") pod \"kserve-controller-manager-d6f4b65b8-fb5cz\" (UID: \"1e7f6181-4f4b-4ef7-a269-b847f43ba563\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:44.499797 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.499763    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:44.512734 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.512708    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:44.652858 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.652827    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"]
Jul 17 18:01:44.653048 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.653028    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc" podUID="e307756f-ae3d-429a-b757-8edd7793a97d" containerName="discovery" containerID="cri-o://86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6" gracePeriod=30
Jul 17 18:01:44.674603 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.669138    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"]
Jul 17 18:01:44.680962 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:01:44.680924    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7f6181_4f4b_4ef7_a269_b847f43ba563.slice/crio-1991b18ea63ca56da84a8f359369810a64988e580bea2a574d07b2c930294e72 WatchSource:0}: Error finding container 1991b18ea63ca56da84a8f359369810a64988e580bea2a574d07b2c930294e72: Status 404 returned error can't find the container with id 1991b18ea63ca56da84a8f359369810a64988e580bea2a574d07b2c930294e72
Jul 17 18:01:44.683099 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.683077    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 18:01:44.702848 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.702817    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"]
Jul 17 18:01:44.704418 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:01:44.704392    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2cb9e2ac_f565_44c3_a832_4a81a349cc4c.slice/crio-d0479eb4525ad5c10d4b1456f9b21cd466686bf4ca930e8c21f5134f0336ef80 WatchSource:0}: Error finding container d0479eb4525ad5c10d4b1456f9b21cd466686bf4ca930e8c21f5134f0336ef80: Status 404 returned error can't find the container with id d0479eb4525ad5c10d4b1456f9b21cd466686bf4ca930e8c21f5134f0336ef80
Jul 17 18:01:44.913999 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.913970    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:44.985426 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.985390    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-istio-token\") pod \"e307756f-ae3d-429a-b757-8edd7793a97d\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") "
Jul 17 18:01:44.985426 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.985424    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-dns-cert\") pod \"e307756f-ae3d-429a-b757-8edd7793a97d\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") "
Jul 17 18:01:44.985710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.985444    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-cacerts\") pod \"e307756f-ae3d-429a-b757-8edd7793a97d\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") "
Jul 17 18:01:44.985710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.985471    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e307756f-ae3d-429a-b757-8edd7793a97d-local-certs\") pod \"e307756f-ae3d-429a-b757-8edd7793a97d\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") "
Jul 17 18:01:44.985710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.985506    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt2dh\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-kube-api-access-pt2dh\") pod \"e307756f-ae3d-429a-b757-8edd7793a97d\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") "
Jul 17 18:01:44.985710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.985542    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-ca-configmap\") pod \"e307756f-ae3d-429a-b757-8edd7793a97d\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") "
Jul 17 18:01:44.985710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.985568    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-kubeconfig\") pod \"e307756f-ae3d-429a-b757-8edd7793a97d\" (UID: \"e307756f-ae3d-429a-b757-8edd7793a97d\") "
Jul 17 18:01:44.986070 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.986022    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e307756f-ae3d-429a-b757-8edd7793a97d" (UID: "e307756f-ae3d-429a-b757-8edd7793a97d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:01:44.988227 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.988182    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-kube-api-access-pt2dh" (OuterVolumeSpecName: "kube-api-access-pt2dh") pod "e307756f-ae3d-429a-b757-8edd7793a97d" (UID: "e307756f-ae3d-429a-b757-8edd7793a97d"). InnerVolumeSpecName "kube-api-access-pt2dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:01:44.988227 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.988194    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e307756f-ae3d-429a-b757-8edd7793a97d-local-certs" (OuterVolumeSpecName: "local-certs") pod "e307756f-ae3d-429a-b757-8edd7793a97d" (UID: "e307756f-ae3d-429a-b757-8edd7793a97d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:01:44.988227 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.988206    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-cacerts" (OuterVolumeSpecName: "cacerts") pod "e307756f-ae3d-429a-b757-8edd7793a97d" (UID: "e307756f-ae3d-429a-b757-8edd7793a97d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:01:44.988227 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.988184    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e307756f-ae3d-429a-b757-8edd7793a97d" (UID: "e307756f-ae3d-429a-b757-8edd7793a97d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:01:44.988488 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.988194    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e307756f-ae3d-429a-b757-8edd7793a97d" (UID: "e307756f-ae3d-429a-b757-8edd7793a97d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:01:44.988488 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:44.988324    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e307756f-ae3d-429a-b757-8edd7793a97d" (UID: "e307756f-ae3d-429a-b757-8edd7793a97d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:01:45.087828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.087746    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-istio-token\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:45.087828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.087770    2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-dns-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:45.087828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.087782    2555 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-cacerts\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:45.087828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.087795    2555 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e307756f-ae3d-429a-b757-8edd7793a97d-local-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:45.087828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.087808    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt2dh\" (UniqueName: \"kubernetes.io/projected/e307756f-ae3d-429a-b757-8edd7793a97d-kube-api-access-pt2dh\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:45.087828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.087820    2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e307756f-ae3d-429a-b757-8edd7793a97d-istio-csr-ca-configmap\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:45.088125 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.087833    2555 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e307756f-ae3d-429a-b757-8edd7793a97d-istio-kubeconfig\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:01:45.447811 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.447722    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg" event={"ID":"2cb9e2ac-f565-44c3-a832-4a81a349cc4c","Type":"ContainerStarted","Data":"d0479eb4525ad5c10d4b1456f9b21cd466686bf4ca930e8c21f5134f0336ef80"}
Jul 17 18:01:45.449042 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.449019    2555 generic.go:356] "Generic (PLEG): container finished" podID="e307756f-ae3d-429a-b757-8edd7793a97d" containerID="86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6" exitCode=0
Jul 17 18:01:45.449165 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.449067    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"
Jul 17 18:01:45.449165 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.449112    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc" event={"ID":"e307756f-ae3d-429a-b757-8edd7793a97d","Type":"ContainerDied","Data":"86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6"}
Jul 17 18:01:45.449165 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.449163    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc" event={"ID":"e307756f-ae3d-429a-b757-8edd7793a97d","Type":"ContainerDied","Data":"c5e1a446dc7c55249f2a874b84cd7a738e776f319cf1d4445fa742d376360975"}
Jul 17 18:01:45.449329 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.449199    2555 scope.go:117] "RemoveContainer" containerID="86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6"
Jul 17 18:01:45.450389 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.450372    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz" event={"ID":"1e7f6181-4f4b-4ef7-a269-b847f43ba563","Type":"ContainerStarted","Data":"1991b18ea63ca56da84a8f359369810a64988e580bea2a574d07b2c930294e72"}
Jul 17 18:01:45.458410 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.458390    2555 scope.go:117] "RemoveContainer" containerID="86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6"
Jul 17 18:01:45.458694 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:01:45.458672    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6\": container with ID starting with 86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6 not found: ID does not exist" containerID="86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6"
Jul 17 18:01:45.458753 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.458705    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6"} err="failed to get container status \"86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6\": rpc error: code = NotFound desc = could not find container \"86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6\": container with ID starting with 86773411fa6aacc6fc83c4fe1ca8194dd966d41ff33b4bf9d99c39a5dd711da6 not found: ID does not exist"
Jul 17 18:01:45.472323 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.472295    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"]
Jul 17 18:01:45.475401 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:45.475374    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-j2sdc"]
Jul 17 18:01:46.945156 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:46.945119    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e307756f-ae3d-429a-b757-8edd7793a97d" path="/var/lib/kubelet/pods/e307756f-ae3d-429a-b757-8edd7793a97d/volumes"
Jul 17 18:01:48.472056 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:48.472014    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz" event={"ID":"1e7f6181-4f4b-4ef7-a269-b847f43ba563","Type":"ContainerStarted","Data":"75925011bf2b5e1594e1a12014b2e4c5837dd851f62cb5468a6856848b069c8c"}
Jul 17 18:01:48.472548 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:48.472149    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:01:48.490901 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:48.490853    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz" podStartSLOduration=2.588209581 podStartE2EDuration="5.490834537s" totalImagesPullingTime="2.902624956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:48.488522299 +0000 UTC m=+306.073998552" watchObservedRunningTime="2026-07-17 18:01:48.490834537 +0000 UTC m=+306.076310794"
Jul 17 18:01:49.478603 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:49.478564    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg" event={"ID":"2cb9e2ac-f565-44c3-a832-4a81a349cc4c","Type":"ContainerStarted","Data":"3e9b8e76945bd5dafe4eabca64a7e372e414b56c77ca2a1a60c30fcc744f91b2"}
Jul 17 18:01:49.479084 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:49.478828    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:01:49.498350 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:49.498285    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg" podStartSLOduration=2.3981868 podStartE2EDuration="6.498270882s" totalImagesPullingTime="4.100084082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:01:49.496412469 +0000 UTC m=+307.081888720" watchObservedRunningTime="2026-07-17 18:01:49.498270882 +0000 UTC m=+307.083747134"
Jul 17 18:01:50.426914 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:50.426883    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-qwmw7"
Jul 17 18:01:50.433129 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:01:50.433103    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dr8fl"
Jul 17 18:02:19.484691 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:19.484661    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fb5cz"
Jul 17 18:02:20.486927 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:20.486893    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cpxwg"
Jul 17 18:02:33.551823 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.551781    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8564"]
Jul 17 18:02:33.552398 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.552322    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e307756f-ae3d-429a-b757-8edd7793a97d" containerName="discovery"
Jul 17 18:02:33.552398 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.552342    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e307756f-ae3d-429a-b757-8edd7793a97d" containerName="discovery"
Jul 17 18:02:33.552513 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.552419    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e307756f-ae3d-429a-b757-8edd7793a97d" containerName="discovery"
Jul 17 18:02:33.566949 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.566913    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k8564"
Jul 17 18:02:33.568819 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.568791    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8564"]
Jul 17 18:02:33.569620 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.569593    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6xwrw\""
Jul 17 18:02:33.569836 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.569815    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 18:02:33.570633 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.570615    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 18:02:33.591363 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.591324    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jffk2\" (UniqueName: \"kubernetes.io/projected/e67f6436-22fa-4aa3-827e-45e05d5b400a-kube-api-access-jffk2\") pod \"kuadrant-operator-catalog-k8564\" (UID: \"e67f6436-22fa-4aa3-827e-45e05d5b400a\") " pod="kuadrant-system/kuadrant-operator-catalog-k8564"
Jul 17 18:02:33.692381 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.692346    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jffk2\" (UniqueName: \"kubernetes.io/projected/e67f6436-22fa-4aa3-827e-45e05d5b400a-kube-api-access-jffk2\") pod \"kuadrant-operator-catalog-k8564\" (UID: \"e67f6436-22fa-4aa3-827e-45e05d5b400a\") " pod="kuadrant-system/kuadrant-operator-catalog-k8564"
Jul 17 18:02:33.700423 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.700400    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jffk2\" (UniqueName: \"kubernetes.io/projected/e67f6436-22fa-4aa3-827e-45e05d5b400a-kube-api-access-jffk2\") pod \"kuadrant-operator-catalog-k8564\" (UID: \"e67f6436-22fa-4aa3-827e-45e05d5b400a\") " pod="kuadrant-system/kuadrant-operator-catalog-k8564"
Jul 17 18:02:33.878170 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.878069    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k8564"
Jul 17 18:02:33.908959 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:33.908926    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8564"]
Jul 17 18:02:34.014668 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.014524    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8564"]
Jul 17 18:02:34.017409 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:02:34.017376    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67f6436_22fa_4aa3_827e_45e05d5b400a.slice/crio-19897d2a246238ee1619c77359dcd94a52029bb90617804acc1bc63dddf23c5b WatchSource:0}: Error finding container 19897d2a246238ee1619c77359dcd94a52029bb90617804acc1bc63dddf23c5b: Status 404 returned error can't find the container with id 19897d2a246238ee1619c77359dcd94a52029bb90617804acc1bc63dddf23c5b
Jul 17 18:02:34.121380 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.118718    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mqfcm"]
Jul 17 18:02:34.123463 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.123424    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:34.130184 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.130116    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mqfcm"]
Jul 17 18:02:34.196750 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.196711    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z46cp\" (UniqueName: \"kubernetes.io/projected/2c4fda1f-ca78-4ff9-a735-de250a40e47f-kube-api-access-z46cp\") pod \"kuadrant-operator-catalog-mqfcm\" (UID: \"2c4fda1f-ca78-4ff9-a735-de250a40e47f\") " pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:34.297352 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.297316    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z46cp\" (UniqueName: \"kubernetes.io/projected/2c4fda1f-ca78-4ff9-a735-de250a40e47f-kube-api-access-z46cp\") pod \"kuadrant-operator-catalog-mqfcm\" (UID: \"2c4fda1f-ca78-4ff9-a735-de250a40e47f\") " pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:34.309664 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.309563    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z46cp\" (UniqueName: \"kubernetes.io/projected/2c4fda1f-ca78-4ff9-a735-de250a40e47f-kube-api-access-z46cp\") pod \"kuadrant-operator-catalog-mqfcm\" (UID: \"2c4fda1f-ca78-4ff9-a735-de250a40e47f\") " pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:34.435548 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.435448    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:34.582812 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.582782    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mqfcm"]
Jul 17 18:02:34.585309 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:02:34.585284    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4fda1f_ca78_4ff9_a735_de250a40e47f.slice/crio-dd70aa83ac2a81c907710937ff4364c25931d1f3b8d8d690e31fbb2d75e147cf WatchSource:0}: Error finding container dd70aa83ac2a81c907710937ff4364c25931d1f3b8d8d690e31fbb2d75e147cf: Status 404 returned error can't find the container with id dd70aa83ac2a81c907710937ff4364c25931d1f3b8d8d690e31fbb2d75e147cf
Jul 17 18:02:34.689500 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.689417    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm" event={"ID":"2c4fda1f-ca78-4ff9-a735-de250a40e47f","Type":"ContainerStarted","Data":"dd70aa83ac2a81c907710937ff4364c25931d1f3b8d8d690e31fbb2d75e147cf"}
Jul 17 18:02:34.690538 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:34.690514    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k8564" event={"ID":"e67f6436-22fa-4aa3-827e-45e05d5b400a","Type":"ContainerStarted","Data":"19897d2a246238ee1619c77359dcd94a52029bb90617804acc1bc63dddf23c5b"}
Jul 17 18:02:36.702974 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:36.702933    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm" event={"ID":"2c4fda1f-ca78-4ff9-a735-de250a40e47f","Type":"ContainerStarted","Data":"d8e5bb1e499c1eb89fc45b2055680eb6469568582c215cd3fd1ce45852a63f9a"}
Jul 17 18:02:36.704409 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:36.704388    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k8564" event={"ID":"e67f6436-22fa-4aa3-827e-45e05d5b400a","Type":"ContainerStarted","Data":"f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae"}
Jul 17 18:02:36.704540 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:36.704432    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-k8564" podUID="e67f6436-22fa-4aa3-827e-45e05d5b400a" containerName="registry-server" containerID="cri-o://f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae" gracePeriod=2
Jul 17 18:02:36.717580 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:36.717533    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm" podStartSLOduration=1.221933133 podStartE2EDuration="2.717515516s" totalImagesPullingTime="1.495582383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:02:36.716338857 +0000 UTC m=+354.301815110" watchObservedRunningTime="2026-07-17 18:02:36.717515516 +0000 UTC m=+354.302991746"
Jul 17 18:02:36.732495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:36.732444    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k8564" podStartSLOduration=1.6672317890000001 podStartE2EDuration="3.732428161s" totalImagesPullingTime="2.065196372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:02:36.730752706 +0000 UTC m=+354.316228982" watchObservedRunningTime="2026-07-17 18:02:36.732428161 +0000 UTC m=+354.317904414"
Jul 17 18:02:36.959688 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:36.959608    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k8564"
Jul 17 18:02:37.024806 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.024767    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jffk2\" (UniqueName: \"kubernetes.io/projected/e67f6436-22fa-4aa3-827e-45e05d5b400a-kube-api-access-jffk2\") pod \"e67f6436-22fa-4aa3-827e-45e05d5b400a\" (UID: \"e67f6436-22fa-4aa3-827e-45e05d5b400a\") "
Jul 17 18:02:37.026950 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.026927    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e67f6436-22fa-4aa3-827e-45e05d5b400a-kube-api-access-jffk2" (OuterVolumeSpecName: "kube-api-access-jffk2") pod "e67f6436-22fa-4aa3-827e-45e05d5b400a" (UID: "e67f6436-22fa-4aa3-827e-45e05d5b400a"). InnerVolumeSpecName "kube-api-access-jffk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:02:37.125880 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.125843    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jffk2\" (UniqueName: \"kubernetes.io/projected/e67f6436-22fa-4aa3-827e-45e05d5b400a-kube-api-access-jffk2\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:37.710484 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.710446    2555 generic.go:356] "Generic (PLEG): container finished" podID="e67f6436-22fa-4aa3-827e-45e05d5b400a" containerID="f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae" exitCode=0
Jul 17 18:02:37.711011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.710500    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k8564"
Jul 17 18:02:37.711011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.710535    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k8564" event={"ID":"e67f6436-22fa-4aa3-827e-45e05d5b400a","Type":"ContainerDied","Data":"f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae"}
Jul 17 18:02:37.711011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.710583    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k8564" event={"ID":"e67f6436-22fa-4aa3-827e-45e05d5b400a","Type":"ContainerDied","Data":"19897d2a246238ee1619c77359dcd94a52029bb90617804acc1bc63dddf23c5b"}
Jul 17 18:02:37.711011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.710606    2555 scope.go:117] "RemoveContainer" containerID="f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae"
Jul 17 18:02:37.719829 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.719811    2555 scope.go:117] "RemoveContainer" containerID="f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae"
Jul 17 18:02:37.720120 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:02:37.720101    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae\": container with ID starting with f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae not found: ID does not exist" containerID="f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae"
Jul 17 18:02:37.720168 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.720130    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae"} err="failed to get container status \"f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae\": rpc error: code = NotFound desc = could not find container \"f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae\": container with ID starting with f5e0d790b5a90a0eefa1cbc52c447bb930b9000129ae5cb85e07e104e8902bae not found: ID does not exist"
Jul 17 18:02:37.730653 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.730623    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8564"]
Jul 17 18:02:37.733940 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:37.733918    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8564"]
Jul 17 18:02:38.943784 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:38.943750    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e67f6436-22fa-4aa3-827e-45e05d5b400a" path="/var/lib/kubelet/pods/e67f6436-22fa-4aa3-827e-45e05d5b400a/volumes"
Jul 17 18:02:44.436456 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:44.436414    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:44.436974 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:44.436480    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:44.459988 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:44.459960    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:44.768551 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:44.768524    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-mqfcm"
Jul 17 18:02:48.952207 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:48.952170    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"]
Jul 17 18:02:48.952581 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:48.952529    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67f6436-22fa-4aa3-827e-45e05d5b400a" containerName="registry-server"
Jul 17 18:02:48.952581 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:48.952539    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67f6436-22fa-4aa3-827e-45e05d5b400a" containerName="registry-server"
Jul 17 18:02:48.952680 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:48.952600    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67f6436-22fa-4aa3-827e-45e05d5b400a" containerName="registry-server"
Jul 17 18:02:48.957417 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:48.957397    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:48.959865 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:48.959845    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-96dnt\""
Jul 17 18:02:48.963620 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:48.963587    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"]
Jul 17 18:02:49.036335 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.036294    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.036335 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.036331    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.036575 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.036428    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpxkb\" (UniqueName: \"kubernetes.io/projected/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-kube-api-access-xpxkb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.137736 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.137692    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpxkb\" (UniqueName: \"kubernetes.io/projected/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-kube-api-access-xpxkb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.137905 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.137788    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.137905 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.137814    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.138175 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.138157    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.138217 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.138201    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.145590 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.145556    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpxkb\" (UniqueName: \"kubernetes.io/projected/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-kube-api-access-xpxkb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.267430 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.267397    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:49.403258 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.403236    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"]
Jul 17 18:02:49.405555 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:02:49.405524    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15b2e38_8b78_4a5a_85d1_cce9e80a793f.slice/crio-bf14e2096fd98f29a2882baa89b9e5279670dc43bf70936c79034f12668c7f8b WatchSource:0}: Error finding container bf14e2096fd98f29a2882baa89b9e5279670dc43bf70936c79034f12668c7f8b: Status 404 returned error can't find the container with id bf14e2096fd98f29a2882baa89b9e5279670dc43bf70936c79034f12668c7f8b
Jul 17 18:02:49.553765 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.553682    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"]
Jul 17 18:02:49.557227 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.557211    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.564341 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.564315    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"]
Jul 17 18:02:49.643146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.643104    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zm2g\" (UniqueName: \"kubernetes.io/projected/9cf51f45-7174-441c-a45f-99db7a138df2-kube-api-access-5zm2g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.643315 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.643214    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.643315 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.643260    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.744618 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.744576    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zm2g\" (UniqueName: \"kubernetes.io/projected/9cf51f45-7174-441c-a45f-99db7a138df2-kube-api-access-5zm2g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.744820 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.744708    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.744820 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.744733    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.745087 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.745071    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.745124 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.745091    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.752978 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.752946    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zm2g\" (UniqueName: \"kubernetes.io/projected/9cf51f45-7174-441c-a45f-99db7a138df2-kube-api-access-5zm2g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.771492 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.771464    2555 generic.go:356] "Generic (PLEG): container finished" podID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerID="d9029d25e3356d64ace8397d9b2b5ac48d65df9cbcc1232e8dd683ab69e46698" exitCode=0
Jul 17 18:02:49.771617 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.771552    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr" event={"ID":"e15b2e38-8b78-4a5a-85d1-cce9e80a793f","Type":"ContainerDied","Data":"d9029d25e3356d64ace8397d9b2b5ac48d65df9cbcc1232e8dd683ab69e46698"}
Jul 17 18:02:49.771617 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.771597    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr" event={"ID":"e15b2e38-8b78-4a5a-85d1-cce9e80a793f","Type":"ContainerStarted","Data":"bf14e2096fd98f29a2882baa89b9e5279670dc43bf70936c79034f12668c7f8b"}
Jul 17 18:02:49.878116 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.878015    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:49.958465 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.958431    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"]
Jul 17 18:02:49.964027 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.964008    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:49.970011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:49.969980    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"]
Jul 17 18:02:50.020797 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.020772    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"]
Jul 17 18:02:50.023072 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:02:50.023045    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf51f45_7174_441c_a45f_99db7a138df2.slice/crio-e7e9b18b7d925c5d869cf1b5883e56720fa9a1b1cde4c48e27d582f61b0811b9 WatchSource:0}: Error finding container e7e9b18b7d925c5d869cf1b5883e56720fa9a1b1cde4c48e27d582f61b0811b9: Status 404 returned error can't find the container with id e7e9b18b7d925c5d869cf1b5883e56720fa9a1b1cde4c48e27d582f61b0811b9
Jul 17 18:02:50.047627 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.047601    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-228l4\" (UniqueName: \"kubernetes.io/projected/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-kube-api-access-228l4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.047750 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.047672    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.047750 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.047702    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.149022 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.148920    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-228l4\" (UniqueName: \"kubernetes.io/projected/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-kube-api-access-228l4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.149022 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.148975    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.149022 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.149004    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.149447 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.149425    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.149523 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.149460    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.157775 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.157746    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-228l4\" (UniqueName: \"kubernetes.io/projected/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-kube-api-access-228l4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.277197 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.277153    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:50.355824 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.355793    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"]
Jul 17 18:02:50.359720 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.359697    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.369615 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.367668    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"]
Jul 17 18:02:50.423199 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.423164    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"]
Jul 17 18:02:50.452489 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.452456    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.452697 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.452503    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9hsh\" (UniqueName: \"kubernetes.io/projected/a25df112-ab04-4732-a67d-c1de4bf0b6af-kube-api-access-k9hsh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.452697 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.452622    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.485723 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:02:50.485686    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb995d59_07b9_4ac8_ade4_db7e3fbfd59f.slice/crio-0d6ecd47f50020fbdaf8612e9799cd4e4465d4883da05fbe542fea083e93cdab WatchSource:0}: Error finding container 0d6ecd47f50020fbdaf8612e9799cd4e4465d4883da05fbe542fea083e93cdab: Status 404 returned error can't find the container with id 0d6ecd47f50020fbdaf8612e9799cd4e4465d4883da05fbe542fea083e93cdab
Jul 17 18:02:50.553260 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.553228    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.553387 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.553326    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.553467 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.553384    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9hsh\" (UniqueName: \"kubernetes.io/projected/a25df112-ab04-4732-a67d-c1de4bf0b6af-kube-api-access-k9hsh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.553690 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.553664    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.553791 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.553754    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.561573 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.561548    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9hsh\" (UniqueName: \"kubernetes.io/projected/a25df112-ab04-4732-a67d-c1de4bf0b6af-kube-api-access-k9hsh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.676172 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.675545    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:50.779069 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.779034    2555 generic.go:356] "Generic (PLEG): container finished" podID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerID="38423e0a26ffb4d3299822df79bb64edc7f6f8520f2e1df0d424630fc0da85fa" exitCode=0
Jul 17 18:02:50.779419 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.779138    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr" event={"ID":"e15b2e38-8b78-4a5a-85d1-cce9e80a793f","Type":"ContainerDied","Data":"38423e0a26ffb4d3299822df79bb64edc7f6f8520f2e1df0d424630fc0da85fa"}
Jul 17 18:02:50.780958 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.780826    2555 generic.go:356] "Generic (PLEG): container finished" podID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerID="39aa33112212aed1deb556ea787f4f7ee274aff74b45c8d919fd695541bb5ef7" exitCode=0
Jul 17 18:02:50.780958 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.780877    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq" event={"ID":"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f","Type":"ContainerDied","Data":"39aa33112212aed1deb556ea787f4f7ee274aff74b45c8d919fd695541bb5ef7"}
Jul 17 18:02:50.780958 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.780922    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq" event={"ID":"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f","Type":"ContainerStarted","Data":"0d6ecd47f50020fbdaf8612e9799cd4e4465d4883da05fbe542fea083e93cdab"}
Jul 17 18:02:50.782853 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.782828    2555 generic.go:356] "Generic (PLEG): container finished" podID="9cf51f45-7174-441c-a45f-99db7a138df2" containerID="561c63e2fd3137794969825d60c9bc8d4f7c4e40fbcd4a39edacdc010aa0f6ab" exitCode=0
Jul 17 18:02:50.782949 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.782900    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46" event={"ID":"9cf51f45-7174-441c-a45f-99db7a138df2","Type":"ContainerDied","Data":"561c63e2fd3137794969825d60c9bc8d4f7c4e40fbcd4a39edacdc010aa0f6ab"}
Jul 17 18:02:50.782949 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.782940    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46" event={"ID":"9cf51f45-7174-441c-a45f-99db7a138df2","Type":"ContainerStarted","Data":"e7e9b18b7d925c5d869cf1b5883e56720fa9a1b1cde4c48e27d582f61b0811b9"}
Jul 17 18:02:50.819516 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:50.819493    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"]
Jul 17 18:02:50.821484 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:02:50.821458    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25df112_ab04_4732_a67d_c1de4bf0b6af.slice/crio-2e78294a337e24ac6c854971a01f7763d856d6f50d2778df8d038b41ceafbcfb WatchSource:0}: Error finding container 2e78294a337e24ac6c854971a01f7763d856d6f50d2778df8d038b41ceafbcfb: Status 404 returned error can't find the container with id 2e78294a337e24ac6c854971a01f7763d856d6f50d2778df8d038b41ceafbcfb
Jul 17 18:02:51.789744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.789704    2555 generic.go:356] "Generic (PLEG): container finished" podID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerID="a1a6b3658be282870077bea0bb04c3652e451cc250df67f11c34b7ad8927e869" exitCode=0
Jul 17 18:02:51.790145 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.789783    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq" event={"ID":"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f","Type":"ContainerDied","Data":"a1a6b3658be282870077bea0bb04c3652e451cc250df67f11c34b7ad8927e869"}
Jul 17 18:02:51.791407 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.791361    2555 generic.go:356] "Generic (PLEG): container finished" podID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerID="7a4f4b698862a267794d783b3750dc2a485986704e9fcf9e46c6d01657867d2b" exitCode=0
Jul 17 18:02:51.791492 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.791437    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x" event={"ID":"a25df112-ab04-4732-a67d-c1de4bf0b6af","Type":"ContainerDied","Data":"7a4f4b698862a267794d783b3750dc2a485986704e9fcf9e46c6d01657867d2b"}
Jul 17 18:02:51.791552 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.791491    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x" event={"ID":"a25df112-ab04-4732-a67d-c1de4bf0b6af","Type":"ContainerStarted","Data":"2e78294a337e24ac6c854971a01f7763d856d6f50d2778df8d038b41ceafbcfb"}
Jul 17 18:02:51.793558 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.793535    2555 generic.go:356] "Generic (PLEG): container finished" podID="9cf51f45-7174-441c-a45f-99db7a138df2" containerID="5a4721c8e0000cafee83bd46db2b0feb4930820d29ac4dd9cfee12e2aa7951f1" exitCode=0
Jul 17 18:02:51.793673 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.793625    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46" event={"ID":"9cf51f45-7174-441c-a45f-99db7a138df2","Type":"ContainerDied","Data":"5a4721c8e0000cafee83bd46db2b0feb4930820d29ac4dd9cfee12e2aa7951f1"}
Jul 17 18:02:51.796141 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.796113    2555 generic.go:356] "Generic (PLEG): container finished" podID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerID="30291526f5720931c1af16c3349a04b45074f5b6549eb5d70812bd0e83bb9f4e" exitCode=0
Jul 17 18:02:51.796245 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:51.796164    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr" event={"ID":"e15b2e38-8b78-4a5a-85d1-cce9e80a793f","Type":"ContainerDied","Data":"30291526f5720931c1af16c3349a04b45074f5b6549eb5d70812bd0e83bb9f4e"}
Jul 17 18:02:52.803566 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:52.803532    2555 generic.go:356] "Generic (PLEG): container finished" podID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerID="5135aa7c3ac24d488796217639a654492c03da64e1940ff084fc5b704f61aa2e" exitCode=0
Jul 17 18:02:52.803964 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:52.803611    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq" event={"ID":"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f","Type":"ContainerDied","Data":"5135aa7c3ac24d488796217639a654492c03da64e1940ff084fc5b704f61aa2e"}
Jul 17 18:02:52.805585 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:52.805556    2555 generic.go:356] "Generic (PLEG): container finished" podID="9cf51f45-7174-441c-a45f-99db7a138df2" containerID="03ccf4b01f6bd402c64b7771713ba4660908433ecf2bcff54db5a28c422ba246" exitCode=0
Jul 17 18:02:52.805726 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:52.805632    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46" event={"ID":"9cf51f45-7174-441c-a45f-99db7a138df2","Type":"ContainerDied","Data":"03ccf4b01f6bd402c64b7771713ba4660908433ecf2bcff54db5a28c422ba246"}
Jul 17 18:02:52.996782 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:52.996752    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:53.080003 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.079901    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-bundle\") pod \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") "
Jul 17 18:02:53.080003 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.079942    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-util\") pod \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") "
Jul 17 18:02:53.080003 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.079985    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpxkb\" (UniqueName: \"kubernetes.io/projected/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-kube-api-access-xpxkb\") pod \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\" (UID: \"e15b2e38-8b78-4a5a-85d1-cce9e80a793f\") "
Jul 17 18:02:53.080468 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.080440    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-bundle" (OuterVolumeSpecName: "bundle") pod "e15b2e38-8b78-4a5a-85d1-cce9e80a793f" (UID: "e15b2e38-8b78-4a5a-85d1-cce9e80a793f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:53.082127 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.082102    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-kube-api-access-xpxkb" (OuterVolumeSpecName: "kube-api-access-xpxkb") pod "e15b2e38-8b78-4a5a-85d1-cce9e80a793f" (UID: "e15b2e38-8b78-4a5a-85d1-cce9e80a793f"). InnerVolumeSpecName "kube-api-access-xpxkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:02:53.085426 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.085390    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-util" (OuterVolumeSpecName: "util") pod "e15b2e38-8b78-4a5a-85d1-cce9e80a793f" (UID: "e15b2e38-8b78-4a5a-85d1-cce9e80a793f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:53.181539 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.181500    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpxkb\" (UniqueName: \"kubernetes.io/projected/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-kube-api-access-xpxkb\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:53.181539 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.181531    2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:53.181539 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.181542    2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15b2e38-8b78-4a5a-85d1-cce9e80a793f-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:53.814554 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.814519    2555 generic.go:356] "Generic (PLEG): container finished" podID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerID="0c54a744a40720b2aa2444618cc9d0582484e214ec92b60c3395fd9f402ed100" exitCode=0
Jul 17 18:02:53.815026 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.814602    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x" event={"ID":"a25df112-ab04-4732-a67d-c1de4bf0b6af","Type":"ContainerDied","Data":"0c54a744a40720b2aa2444618cc9d0582484e214ec92b60c3395fd9f402ed100"}
Jul 17 18:02:53.816705 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.816675    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr" event={"ID":"e15b2e38-8b78-4a5a-85d1-cce9e80a793f","Type":"ContainerDied","Data":"bf14e2096fd98f29a2882baa89b9e5279670dc43bf70936c79034f12668c7f8b"}
Jul 17 18:02:53.816785 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.816722    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf14e2096fd98f29a2882baa89b9e5279670dc43bf70936c79034f12668c7f8b"
Jul 17 18:02:53.816903 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.816889    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr"
Jul 17 18:02:53.977557 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:53.977528    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:54.045025 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.044996    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:54.089919 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.089825    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-228l4\" (UniqueName: \"kubernetes.io/projected/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-kube-api-access-228l4\") pod \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") "
Jul 17 18:02:54.089919 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.089907    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-util\") pod \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") "
Jul 17 18:02:54.090148 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.090024    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-bundle\") pod \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\" (UID: \"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f\") "
Jul 17 18:02:54.090524 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.090494    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-bundle" (OuterVolumeSpecName: "bundle") pod "fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" (UID: "fb995d59-07b9-4ac8-ade4-db7e3fbfd59f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:54.092007 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.091988    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-kube-api-access-228l4" (OuterVolumeSpecName: "kube-api-access-228l4") pod "fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" (UID: "fb995d59-07b9-4ac8-ade4-db7e3fbfd59f"). InnerVolumeSpecName "kube-api-access-228l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:02:54.095191 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.095166    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-util" (OuterVolumeSpecName: "util") pod "fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" (UID: "fb995d59-07b9-4ac8-ade4-db7e3fbfd59f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:54.191305 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.191267    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zm2g\" (UniqueName: \"kubernetes.io/projected/9cf51f45-7174-441c-a45f-99db7a138df2-kube-api-access-5zm2g\") pod \"9cf51f45-7174-441c-a45f-99db7a138df2\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") "
Jul 17 18:02:54.191482 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.191317    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-bundle\") pod \"9cf51f45-7174-441c-a45f-99db7a138df2\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") "
Jul 17 18:02:54.191482 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.191349    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-util\") pod \"9cf51f45-7174-441c-a45f-99db7a138df2\" (UID: \"9cf51f45-7174-441c-a45f-99db7a138df2\") "
Jul 17 18:02:54.191593 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.191542    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-228l4\" (UniqueName: \"kubernetes.io/projected/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-kube-api-access-228l4\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:54.191593 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.191552    2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:54.191593 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.191563    2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb995d59-07b9-4ac8-ade4-db7e3fbfd59f-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:54.192012 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.191987    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-bundle" (OuterVolumeSpecName: "bundle") pod "9cf51f45-7174-441c-a45f-99db7a138df2" (UID: "9cf51f45-7174-441c-a45f-99db7a138df2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:54.193495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.193474    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf51f45-7174-441c-a45f-99db7a138df2-kube-api-access-5zm2g" (OuterVolumeSpecName: "kube-api-access-5zm2g") pod "9cf51f45-7174-441c-a45f-99db7a138df2" (UID: "9cf51f45-7174-441c-a45f-99db7a138df2"). InnerVolumeSpecName "kube-api-access-5zm2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:02:54.197117 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.197089    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-util" (OuterVolumeSpecName: "util") pod "9cf51f45-7174-441c-a45f-99db7a138df2" (UID: "9cf51f45-7174-441c-a45f-99db7a138df2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:54.292930 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.292888    2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:54.292930 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.292923    2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cf51f45-7174-441c-a45f-99db7a138df2-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:54.292930 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.292935    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zm2g\" (UniqueName: \"kubernetes.io/projected/9cf51f45-7174-441c-a45f-99db7a138df2-kube-api-access-5zm2g\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:54.823805 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.823779    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq"
Jul 17 18:02:54.824246 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.823778    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq" event={"ID":"fb995d59-07b9-4ac8-ade4-db7e3fbfd59f","Type":"ContainerDied","Data":"0d6ecd47f50020fbdaf8612e9799cd4e4465d4883da05fbe542fea083e93cdab"}
Jul 17 18:02:54.824246 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.823895    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d6ecd47f50020fbdaf8612e9799cd4e4465d4883da05fbe542fea083e93cdab"
Jul 17 18:02:54.825971 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.825946    2555 generic.go:356] "Generic (PLEG): container finished" podID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerID="0aadbba18fd3d533e5eb8787c8387774e47ebf476bf039d9cee6e7bdd827f9c0" exitCode=0
Jul 17 18:02:54.826098 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.826021    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x" event={"ID":"a25df112-ab04-4732-a67d-c1de4bf0b6af","Type":"ContainerDied","Data":"0aadbba18fd3d533e5eb8787c8387774e47ebf476bf039d9cee6e7bdd827f9c0"}
Jul 17 18:02:54.827864 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.827845    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46"
Jul 17 18:02:54.827946 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.827845    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46" event={"ID":"9cf51f45-7174-441c-a45f-99db7a138df2","Type":"ContainerDied","Data":"e7e9b18b7d925c5d869cf1b5883e56720fa9a1b1cde4c48e27d582f61b0811b9"}
Jul 17 18:02:54.827997 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:54.827969    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7e9b18b7d925c5d869cf1b5883e56720fa9a1b1cde4c48e27d582f61b0811b9"
Jul 17 18:02:55.973481 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:55.973455    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:56.109835 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.109745    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-bundle\") pod \"a25df112-ab04-4732-a67d-c1de4bf0b6af\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") "
Jul 17 18:02:56.109835 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.109804    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9hsh\" (UniqueName: \"kubernetes.io/projected/a25df112-ab04-4732-a67d-c1de4bf0b6af-kube-api-access-k9hsh\") pod \"a25df112-ab04-4732-a67d-c1de4bf0b6af\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") "
Jul 17 18:02:56.110046 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.109857    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-util\") pod \"a25df112-ab04-4732-a67d-c1de4bf0b6af\" (UID: \"a25df112-ab04-4732-a67d-c1de4bf0b6af\") "
Jul 17 18:02:56.110292 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.110262    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-bundle" (OuterVolumeSpecName: "bundle") pod "a25df112-ab04-4732-a67d-c1de4bf0b6af" (UID: "a25df112-ab04-4732-a67d-c1de4bf0b6af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:56.110434 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.110339    2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:56.112103 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.112079    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25df112-ab04-4732-a67d-c1de4bf0b6af-kube-api-access-k9hsh" (OuterVolumeSpecName: "kube-api-access-k9hsh") pod "a25df112-ab04-4732-a67d-c1de4bf0b6af" (UID: "a25df112-ab04-4732-a67d-c1de4bf0b6af"). InnerVolumeSpecName "kube-api-access-k9hsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:02:56.190453 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.190405    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-util" (OuterVolumeSpecName: "util") pod "a25df112-ab04-4732-a67d-c1de4bf0b6af" (UID: "a25df112-ab04-4732-a67d-c1de4bf0b6af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:02:56.211691 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.211633    2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a25df112-ab04-4732-a67d-c1de4bf0b6af-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:56.211790 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.211693    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9hsh\" (UniqueName: \"kubernetes.io/projected/a25df112-ab04-4732-a67d-c1de4bf0b6af-kube-api-access-k9hsh\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:02:56.840070 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.840041    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x"
Jul 17 18:02:56.840234 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.840039    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x" event={"ID":"a25df112-ab04-4732-a67d-c1de4bf0b6af","Type":"ContainerDied","Data":"2e78294a337e24ac6c854971a01f7763d856d6f50d2778df8d038b41ceafbcfb"}
Jul 17 18:02:56.840234 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:02:56.840160    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e78294a337e24ac6c854971a01f7763d856d6f50d2778df8d038b41ceafbcfb"
Jul 17 18:03:05.822515 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822481    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"]
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822856    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822868    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822875    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf51f45-7174-441c-a45f-99db7a138df2" containerName="extract"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822881    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf51f45-7174-441c-a45f-99db7a138df2" containerName="extract"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822893    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf51f45-7174-441c-a45f-99db7a138df2" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822900    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf51f45-7174-441c-a45f-99db7a138df2" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822907    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerName="extract"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822912    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerName="extract"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822921    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822926    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822937    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822942    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822951    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf51f45-7174-441c-a45f-99db7a138df2" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822955    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf51f45-7174-441c-a45f-99db7a138df2" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822963    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822967    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerName="util"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822972    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822977    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822985    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerName="extract"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822991    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerName="extract"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.822997    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.823002    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerName="pull"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.823007    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerName="extract"
Jul 17 18:03:05.823036 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.823012    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerName="extract"
Jul 17 18:03:05.823796 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.823064    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf51f45-7174-441c-a45f-99db7a138df2" containerName="extract"
Jul 17 18:03:05.823796 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.823075    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a25df112-ab04-4732-a67d-c1de4bf0b6af" containerName="extract"
Jul 17 18:03:05.823796 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.823082    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e15b2e38-8b78-4a5a-85d1-cce9e80a793f" containerName="extract"
Jul 17 18:03:05.823796 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.823087    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb995d59-07b9-4ac8-ade4-db7e3fbfd59f" containerName="extract"
Jul 17 18:03:05.827370 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.827352    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"
Jul 17 18:03:05.830436 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.830410    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5952r\""
Jul 17 18:03:05.838499 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.838418    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"]
Jul 17 18:03:05.901240 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:05.901200    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjlb6\" (UniqueName: \"kubernetes.io/projected/4595cf40-0d56-4b29-902b-2a74306cd93c-kube-api-access-jjlb6\") pod \"authorino-operator-75cc8d5cb7-22gvf\" (UID: \"4595cf40-0d56-4b29-902b-2a74306cd93c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"
Jul 17 18:03:06.002289 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:06.002249    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjlb6\" (UniqueName: \"kubernetes.io/projected/4595cf40-0d56-4b29-902b-2a74306cd93c-kube-api-access-jjlb6\") pod \"authorino-operator-75cc8d5cb7-22gvf\" (UID: \"4595cf40-0d56-4b29-902b-2a74306cd93c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"
Jul 17 18:03:06.020472 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:06.020437    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjlb6\" (UniqueName: \"kubernetes.io/projected/4595cf40-0d56-4b29-902b-2a74306cd93c-kube-api-access-jjlb6\") pod \"authorino-operator-75cc8d5cb7-22gvf\" (UID: \"4595cf40-0d56-4b29-902b-2a74306cd93c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"
Jul 17 18:03:06.140116 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:06.140029    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"
Jul 17 18:03:06.278087 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:06.278062    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"]
Jul 17 18:03:06.280288 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:06.280263    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4595cf40_0d56_4b29_902b_2a74306cd93c.slice/crio-e9a4156f3e430e06d65552407d386675be7ae6ff9bf914126ae5173e0fa43b8b WatchSource:0}: Error finding container e9a4156f3e430e06d65552407d386675be7ae6ff9bf914126ae5173e0fa43b8b: Status 404 returned error can't find the container with id e9a4156f3e430e06d65552407d386675be7ae6ff9bf914126ae5173e0fa43b8b
Jul 17 18:03:06.892519 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:06.892485    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf" event={"ID":"4595cf40-0d56-4b29-902b-2a74306cd93c","Type":"ContainerStarted","Data":"e9a4156f3e430e06d65552407d386675be7ae6ff9bf914126ae5173e0fa43b8b"}
Jul 17 18:03:07.454732 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.454695    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b47b969d6-rtq7j"]
Jul 17 18:03:07.458626 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.458602    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.472698 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.472667    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b47b969d6-rtq7j"]
Jul 17 18:03:07.513907 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.513842    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-config\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.513907 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.513897    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-trusted-ca-bundle\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.514157 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.513926    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-oauth-serving-cert\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.514157 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.513977    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-service-ca\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.514157 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.514022    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-oauth-config\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.514157 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.514045    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dst5r\" (UniqueName: \"kubernetes.io/projected/c61aee4b-3b03-4849-b944-a39ee9c9163b-kube-api-access-dst5r\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.514157 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.514106    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-serving-cert\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.614624 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.614581    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-oauth-config\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.614624 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.614622    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dst5r\" (UniqueName: \"kubernetes.io/projected/c61aee4b-3b03-4849-b944-a39ee9c9163b-kube-api-access-dst5r\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.614873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.614659    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-serving-cert\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.614873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.614715    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-config\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.614873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.614742    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-trusted-ca-bundle\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.614873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.614768    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-oauth-serving-cert\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.614873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.614817    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-service-ca\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.615586 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.615531    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-config\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.615586 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.615561    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-service-ca\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.615804 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.615561    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-oauth-serving-cert\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.615946 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.615915    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c61aee4b-3b03-4849-b944-a39ee9c9163b-trusted-ca-bundle\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.617608 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.617587    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-oauth-config\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.617834 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.617814    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c61aee4b-3b03-4849-b944-a39ee9c9163b-console-serving-cert\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.622513 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.622485    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dst5r\" (UniqueName: \"kubernetes.io/projected/c61aee4b-3b03-4849-b944-a39ee9c9163b-kube-api-access-dst5r\") pod \"console-6b47b969d6-rtq7j\" (UID: \"c61aee4b-3b03-4849-b944-a39ee9c9163b\") " pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:07.770670 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:07.770626    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:08.658313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:08.658286    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b47b969d6-rtq7j"]
Jul 17 18:03:08.660093 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:08.660064    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61aee4b_3b03_4849_b944_a39ee9c9163b.slice/crio-4ed36961e2ce6fd94116774b7a880e08aa073bf2eeb73bf7cb83e4c5d19a59bb WatchSource:0}: Error finding container 4ed36961e2ce6fd94116774b7a880e08aa073bf2eeb73bf7cb83e4c5d19a59bb: Status 404 returned error can't find the container with id 4ed36961e2ce6fd94116774b7a880e08aa073bf2eeb73bf7cb83e4c5d19a59bb
Jul 17 18:03:08.905961 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:08.905866    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf" event={"ID":"4595cf40-0d56-4b29-902b-2a74306cd93c","Type":"ContainerStarted","Data":"f0e46e15ea9bf685b2eeb101a8c112442aa14e859b6755a6cf6895c4d910de86"}
Jul 17 18:03:08.906135 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:08.905987    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"
Jul 17 18:03:08.907509 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:08.907487    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b47b969d6-rtq7j" event={"ID":"c61aee4b-3b03-4849-b944-a39ee9c9163b","Type":"ContainerStarted","Data":"6ab6c08081c911cf8b54bc22882d8202d237771ca4bac530cb83e22a24bf85ab"}
Jul 17 18:03:08.907620 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:08.907521    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b47b969d6-rtq7j" event={"ID":"c61aee4b-3b03-4849-b944-a39ee9c9163b","Type":"ContainerStarted","Data":"4ed36961e2ce6fd94116774b7a880e08aa073bf2eeb73bf7cb83e4c5d19a59bb"}
Jul 17 18:03:08.931144 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:08.931097    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf" podStartSLOduration=1.650877925 podStartE2EDuration="3.931079892s" totalImagesPullingTime="2.280201967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:08.929189606 +0000 UTC m=+386.514665859" watchObservedRunningTime="2026-07-17 18:03:08.931079892 +0000 UTC m=+386.516556143"
Jul 17 18:03:08.952493 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:08.952439    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b47b969d6-rtq7j" podStartSLOduration=1.952420879 podStartE2EDuration="1.952420879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:08.950861473 +0000 UTC m=+386.536337719" watchObservedRunningTime="2026-07-17 18:03:08.952420879 +0000 UTC m=+386.537897128"
Jul 17 18:03:10.969361 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:10.969323    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"]
Jul 17 18:03:10.977772 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:10.977747    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:10.980150 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:10.980121    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nbkwq\""
Jul 17 18:03:10.990107 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:10.988447    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"]
Jul 17 18:03:11.047402 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.047349    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f696906f-eaaa-4780-b1ed-a0341ecdefc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:11.047593 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.047434    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hgz5\" (UniqueName: \"kubernetes.io/projected/f696906f-eaaa-4780-b1ed-a0341ecdefc7-kube-api-access-5hgz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:11.148029 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.147977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f696906f-eaaa-4780-b1ed-a0341ecdefc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:11.148192 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.148061    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hgz5\" (UniqueName: \"kubernetes.io/projected/f696906f-eaaa-4780-b1ed-a0341ecdefc7-kube-api-access-5hgz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:11.148399 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.148376    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f696906f-eaaa-4780-b1ed-a0341ecdefc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:11.156448 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.156422    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hgz5\" (UniqueName: \"kubernetes.io/projected/f696906f-eaaa-4780-b1ed-a0341ecdefc7-kube-api-access-5hgz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:11.291809 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.291767    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:11.434035 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.434008    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"]
Jul 17 18:03:11.436445 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:11.436411    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf696906f_eaaa_4780_b1ed_a0341ecdefc7.slice/crio-bc6086a932530fd9ca55770c917e0934eaf15cbbaa33656353c71b5a522b637d WatchSource:0}: Error finding container bc6086a932530fd9ca55770c917e0934eaf15cbbaa33656353c71b5a522b637d: Status 404 returned error can't find the container with id bc6086a932530fd9ca55770c917e0934eaf15cbbaa33656353c71b5a522b637d
Jul 17 18:03:11.926299 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:11.926259    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph" event={"ID":"f696906f-eaaa-4780-b1ed-a0341ecdefc7","Type":"ContainerStarted","Data":"bc6086a932530fd9ca55770c917e0934eaf15cbbaa33656353c71b5a522b637d"}
Jul 17 18:03:13.700194 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.700158    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"]
Jul 17 18:03:13.703992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.703974    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"
Jul 17 18:03:13.706596 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.706572    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-p87l4\""
Jul 17 18:03:13.706596 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.706572    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 17 18:03:13.713916 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.713889    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"]
Jul 17 18:03:13.773028 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.772991    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsdvz\" (UniqueName: \"kubernetes.io/projected/b929e9a6-fb69-4ce2-9605-20b21c4f9515-kube-api-access-jsdvz\") pod \"dns-operator-controller-manager-864dd78c4c-m6fdq\" (UID: \"b929e9a6-fb69-4ce2-9605-20b21c4f9515\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"
Jul 17 18:03:13.874428 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.874394    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsdvz\" (UniqueName: \"kubernetes.io/projected/b929e9a6-fb69-4ce2-9605-20b21c4f9515-kube-api-access-jsdvz\") pod \"dns-operator-controller-manager-864dd78c4c-m6fdq\" (UID: \"b929e9a6-fb69-4ce2-9605-20b21c4f9515\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"
Jul 17 18:03:13.887947 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:13.887914    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsdvz\" (UniqueName: \"kubernetes.io/projected/b929e9a6-fb69-4ce2-9605-20b21c4f9515-kube-api-access-jsdvz\") pod \"dns-operator-controller-manager-864dd78c4c-m6fdq\" (UID: \"b929e9a6-fb69-4ce2-9605-20b21c4f9515\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"
Jul 17 18:03:14.015939 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:14.015900    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"
Jul 17 18:03:14.202197 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:14.202174    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"]
Jul 17 18:03:14.203886 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:14.203848    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb929e9a6_fb69_4ce2_9605_20b21c4f9515.slice/crio-d8f2d5cb59ce05a1782c49150f8801d34d9f2e33e509fc4b7090c3695d849cf9 WatchSource:0}: Error finding container d8f2d5cb59ce05a1782c49150f8801d34d9f2e33e509fc4b7090c3695d849cf9: Status 404 returned error can't find the container with id d8f2d5cb59ce05a1782c49150f8801d34d9f2e33e509fc4b7090c3695d849cf9
Jul 17 18:03:14.946779 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:14.946742    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq" event={"ID":"b929e9a6-fb69-4ce2-9605-20b21c4f9515","Type":"ContainerStarted","Data":"d8f2d5cb59ce05a1782c49150f8801d34d9f2e33e509fc4b7090c3695d849cf9"}
Jul 17 18:03:16.271767 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.271737    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"]
Jul 17 18:03:16.276831 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.276810    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:16.279383 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.279363    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j8kkj\""
Jul 17 18:03:16.290593 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.287580    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"]
Jul 17 18:03:16.394758 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.394714    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2vkt\" (UniqueName: \"kubernetes.io/projected/551208cf-59c7-4400-8473-e3438ef0d16a-kube-api-access-g2vkt\") pod \"limitador-operator-controller-manager-859d7b78bd-krqfd\" (UID: \"551208cf-59c7-4400-8473-e3438ef0d16a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:16.495655 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.495610    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2vkt\" (UniqueName: \"kubernetes.io/projected/551208cf-59c7-4400-8473-e3438ef0d16a-kube-api-access-g2vkt\") pod \"limitador-operator-controller-manager-859d7b78bd-krqfd\" (UID: \"551208cf-59c7-4400-8473-e3438ef0d16a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:16.505170 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.505134    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2vkt\" (UniqueName: \"kubernetes.io/projected/551208cf-59c7-4400-8473-e3438ef0d16a-kube-api-access-g2vkt\") pod \"limitador-operator-controller-manager-859d7b78bd-krqfd\" (UID: \"551208cf-59c7-4400-8473-e3438ef0d16a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:16.602559 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.602524    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:16.961766 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.961673    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq" event={"ID":"b929e9a6-fb69-4ce2-9605-20b21c4f9515","Type":"ContainerStarted","Data":"35836b1b62b29c03446b6137177851915dc82aeeaf98391631514b9d11ec2d6e"}
Jul 17 18:03:16.961923 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.961772    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"
Jul 17 18:03:16.979655 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:16.979587    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq" podStartSLOduration=1.945707861 podStartE2EDuration="3.979564446s" totalImagesPullingTime="2.033856585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:16.978799123 +0000 UTC m=+394.564275376" watchObservedRunningTime="2026-07-17 18:03:16.979564446 +0000 UTC m=+394.565040683"
Jul 17 18:03:17.611830 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.611805    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"]
Jul 17 18:03:17.614068 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:17.614045    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551208cf_59c7_4400_8473_e3438ef0d16a.slice/crio-7e02024087aac7bd3a1321fd685a83e6dc151079d6eacb6c5efabbeecce20ff6 WatchSource:0}: Error finding container 7e02024087aac7bd3a1321fd685a83e6dc151079d6eacb6c5efabbeecce20ff6: Status 404 returned error can't find the container with id 7e02024087aac7bd3a1321fd685a83e6dc151079d6eacb6c5efabbeecce20ff6
Jul 17 18:03:17.771182 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.771145    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:17.771364 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.771197    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:17.775864 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.775843    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:17.969228 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.969187    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph" event={"ID":"f696906f-eaaa-4780-b1ed-a0341ecdefc7","Type":"ContainerStarted","Data":"67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b"}
Jul 17 18:03:17.969425 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.969294    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:17.970489 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.970468    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd" event={"ID":"551208cf-59c7-4400-8473-e3438ef0d16a","Type":"ContainerStarted","Data":"7e02024087aac7bd3a1321fd685a83e6dc151079d6eacb6c5efabbeecce20ff6"}
Jul 17 18:03:17.974299 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.974281    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b47b969d6-rtq7j"
Jul 17 18:03:17.988226 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:17.988183    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph" podStartSLOduration=1.886742103 podStartE2EDuration="7.988169178s" totalImagesPullingTime="6.101427075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:17.987321227 +0000 UTC m=+395.572797480" watchObservedRunningTime="2026-07-17 18:03:17.988169178 +0000 UTC m=+395.573645429"
Jul 17 18:03:18.034933 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:18.034832    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846f4ff7b6-m6v4l"]
Jul 17 18:03:19.915874 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:19.915843    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22gvf"
Jul 17 18:03:19.984493 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:19.984450    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd" event={"ID":"551208cf-59c7-4400-8473-e3438ef0d16a","Type":"ContainerStarted","Data":"5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826"}
Jul 17 18:03:19.984705 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:19.984613    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:20.005982 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.005928    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd" podStartSLOduration=2.428152932 podStartE2EDuration="4.005911613s" totalImagesPullingTime="1.577758681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:20.004475693 +0000 UTC m=+397.589951945" watchObservedRunningTime="2026-07-17 18:03:20.005911613 +0000 UTC m=+397.591387865"
Jul 17 18:03:20.354112 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.354076    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"]
Jul 17 18:03:20.358044 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.358020    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.360720 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.360698    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 17 18:03:20.360853 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.360768    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 17 18:03:20.360853 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.360781    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-96dnt\""
Jul 17 18:03:20.370197 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.370172    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"]
Jul 17 18:03:20.433457 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.433422    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc78a7d7-9035-4cd4-b12e-56642e234855-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.433633 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.433473    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djv6v\" (UniqueName: \"kubernetes.io/projected/fc78a7d7-9035-4cd4-b12e-56642e234855-kube-api-access-djv6v\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.433633 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.433496    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc78a7d7-9035-4cd4-b12e-56642e234855-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.534126 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.534085    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc78a7d7-9035-4cd4-b12e-56642e234855-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.534320 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.534138    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djv6v\" (UniqueName: \"kubernetes.io/projected/fc78a7d7-9035-4cd4-b12e-56642e234855-kube-api-access-djv6v\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.534320 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.534160    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc78a7d7-9035-4cd4-b12e-56642e234855-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.534811 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.534793    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc78a7d7-9035-4cd4-b12e-56642e234855-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.536474 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.536453    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc78a7d7-9035-4cd4-b12e-56642e234855-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.547675 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.547624    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djv6v\" (UniqueName: \"kubernetes.io/projected/fc78a7d7-9035-4cd4-b12e-56642e234855-kube-api-access-djv6v\") pod \"kuadrant-console-plugin-66b5774985-xnbfk\" (UID: \"fc78a7d7-9035-4cd4-b12e-56642e234855\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.666896 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.666805    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"
Jul 17 18:03:20.812413 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.812387    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk"]
Jul 17 18:03:20.814943 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:20.814907    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc78a7d7_9035_4cd4_b12e_56642e234855.slice/crio-2f31896e99dc8e22c39210066010ed2008ed9fc3b7204a5379d5a1771e0059e0 WatchSource:0}: Error finding container 2f31896e99dc8e22c39210066010ed2008ed9fc3b7204a5379d5a1771e0059e0: Status 404 returned error can't find the container with id 2f31896e99dc8e22c39210066010ed2008ed9fc3b7204a5379d5a1771e0059e0
Jul 17 18:03:20.991577 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:20.991549    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk" event={"ID":"fc78a7d7-9035-4cd4-b12e-56642e234855","Type":"ContainerStarted","Data":"2f31896e99dc8e22c39210066010ed2008ed9fc3b7204a5379d5a1771e0059e0"}
Jul 17 18:03:27.973376 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:27.973341    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m6fdq"
Jul 17 18:03:28.980111 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:28.980077    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:30.201490 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.201450    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"]
Jul 17 18:03:30.212619 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.212590    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.216266 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.216237    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"]
Jul 17 18:03:30.329290 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.329255    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdm5f\" (UniqueName: \"kubernetes.io/projected/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-kube-api-access-fdm5f\") pod \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.329486 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.329415    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.430830 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.430785    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.430830 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.430837    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdm5f\" (UniqueName: \"kubernetes.io/projected/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-kube-api-access-fdm5f\") pod \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.431233 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.431214    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.440863 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.440832    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdm5f\" (UniqueName: \"kubernetes.io/projected/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-kube-api-access-fdm5f\") pod \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.528328 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.528286    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:30.647262 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.647221    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"]
Jul 17 18:03:30.653890 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.653851    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"]
Jul 17 18:03:30.674013 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.673979    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"]
Jul 17 18:03:30.681481 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.681455    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:30.696121 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.696091    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"]
Jul 17 18:03:30.702732 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.702699    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"]
Jul 17 18:03:30.702948 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.702923    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" containerName="manager" containerID="cri-o://67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b" gracePeriod=2
Jul 17 18:03:30.718197 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.718154    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"]
Jul 17 18:03:30.730151 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.729755    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"]
Jul 17 18:03:30.730329 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.730134    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" containerName="manager" containerID="cri-o://5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826" gracePeriod=2
Jul 17 18:03:30.733243 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:30.733197    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-krqfd\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:30.733802 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.733589    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktsjv\" (UniqueName: \"kubernetes.io/projected/057591ea-b18e-43ce-897c-371e1a89be63-kube-api-access-ktsjv\") pod \"kuadrant-operator-controller-manager-55467d9f7-pnvdx\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:30.733802 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.733715    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057591ea-b18e-43ce-897c-371e1a89be63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pnvdx\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:30.733975 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.733924    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"]
Jul 17 18:03:30.735065 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:30.735036    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:30.754520 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.754484    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"]
Jul 17 18:03:30.755147 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.755124    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" containerName="manager"
Jul 17 18:03:30.755147 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.755147    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" containerName="manager"
Jul 17 18:03:30.755339 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.755161    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" containerName="manager"
Jul 17 18:03:30.755339 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.755169    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" containerName="manager"
Jul 17 18:03:30.755443 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.755340    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" containerName="manager"
Jul 17 18:03:30.755443 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.755360    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" containerName="manager"
Jul 17 18:03:30.759119 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.759097    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"
Jul 17 18:03:30.769723 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.769682    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"]
Jul 17 18:03:30.780577 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:30.780485    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-krqfd\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:30.782449 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:30.782406    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:30.834591 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.834548    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057591ea-b18e-43ce-897c-371e1a89be63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pnvdx\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:30.834810 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.834612    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktsjv\" (UniqueName: \"kubernetes.io/projected/057591ea-b18e-43ce-897c-371e1a89be63-kube-api-access-ktsjv\") pod \"kuadrant-operator-controller-manager-55467d9f7-pnvdx\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:30.834810 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.834653    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glgcv\" (UniqueName: \"kubernetes.io/projected/8295f3fd-3c4f-4cdd-8685-087c250085b4-kube-api-access-glgcv\") pod \"limitador-operator-controller-manager-859d7b78bd-2rjpk\" (UID: \"8295f3fd-3c4f-4cdd-8685-087c250085b4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"
Jul 17 18:03:30.835018 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.834997    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057591ea-b18e-43ce-897c-371e1a89be63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pnvdx\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:30.846076 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.846045    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktsjv\" (UniqueName: \"kubernetes.io/projected/057591ea-b18e-43ce-897c-371e1a89be63-kube-api-access-ktsjv\") pod \"kuadrant-operator-controller-manager-55467d9f7-pnvdx\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:30.936010 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.935976    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glgcv\" (UniqueName: \"kubernetes.io/projected/8295f3fd-3c4f-4cdd-8685-087c250085b4-kube-api-access-glgcv\") pod \"limitador-operator-controller-manager-859d7b78bd-2rjpk\" (UID: \"8295f3fd-3c4f-4cdd-8685-087c250085b4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"
Jul 17 18:03:30.945562 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.945526    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glgcv\" (UniqueName: \"kubernetes.io/projected/8295f3fd-3c4f-4cdd-8685-087c250085b4-kube-api-access-glgcv\") pod \"limitador-operator-controller-manager-859d7b78bd-2rjpk\" (UID: \"8295f3fd-3c4f-4cdd-8685-087c250085b4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"
Jul 17 18:03:30.995606 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:30.995568    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:31.125389 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:31.125304    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"
Jul 17 18:03:43.062485 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.062416    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-846f4ff7b6-m6v4l" podUID="8aa9b44c-6a8a-4114-9a03-52ee1e053308" containerName="console" containerID="cri-o://3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4" gracePeriod=15
Jul 17 18:03:43.326454 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.326063    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:43.328467 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:43.328420    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-krqfd\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:43.375873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.375845    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:43.378084 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:43.378039    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-krqfd\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:43.379893 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:43.379865    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:43.393629 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.393603    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 18:03:43.395675 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:43.395652    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-krqfd\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:43.397525 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:43.397505    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:43.455719 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.455684    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hgz5\" (UniqueName: \"kubernetes.io/projected/f696906f-eaaa-4780-b1ed-a0341ecdefc7-kube-api-access-5hgz5\") pod \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") "
Jul 17 18:03:43.455927 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.455794    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f696906f-eaaa-4780-b1ed-a0341ecdefc7-extensions-socket-volume\") pod \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\" (UID: \"f696906f-eaaa-4780-b1ed-a0341ecdefc7\") "
Jul 17 18:03:43.455927 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.455835    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2vkt\" (UniqueName: \"kubernetes.io/projected/551208cf-59c7-4400-8473-e3438ef0d16a-kube-api-access-g2vkt\") pod \"551208cf-59c7-4400-8473-e3438ef0d16a\" (UID: \"551208cf-59c7-4400-8473-e3438ef0d16a\") "
Jul 17 18:03:43.456310 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.456275    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f696906f-eaaa-4780-b1ed-a0341ecdefc7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f696906f-eaaa-4780-b1ed-a0341ecdefc7" (UID: "f696906f-eaaa-4780-b1ed-a0341ecdefc7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:03:43.457867 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.457846    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f696906f-eaaa-4780-b1ed-a0341ecdefc7-kube-api-access-5hgz5" (OuterVolumeSpecName: "kube-api-access-5hgz5") pod "f696906f-eaaa-4780-b1ed-a0341ecdefc7" (UID: "f696906f-eaaa-4780-b1ed-a0341ecdefc7"). InnerVolumeSpecName "kube-api-access-5hgz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:03:43.457867 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.457861    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/551208cf-59c7-4400-8473-e3438ef0d16a-kube-api-access-g2vkt" (OuterVolumeSpecName: "kube-api-access-g2vkt") pod "551208cf-59c7-4400-8473-e3438ef0d16a" (UID: "551208cf-59c7-4400-8473-e3438ef0d16a"). InnerVolumeSpecName "kube-api-access-g2vkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:03:43.557201 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557169    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk4xv\" (UniqueName: \"kubernetes.io/projected/8aa9b44c-6a8a-4114-9a03-52ee1e053308-kube-api-access-wk4xv\") pod \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") "
Jul 17 18:03:43.557393 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557216    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-trusted-ca-bundle\") pod \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") "
Jul 17 18:03:43.557393 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557250    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-oauth-config\") pod \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") "
Jul 17 18:03:43.557393 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557277    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-serving-cert\") pod \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") "
Jul 17 18:03:43.557560 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557395    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-service-ca\") pod \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") "
Jul 17 18:03:43.557560 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557458    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-config\") pod \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") "
Jul 17 18:03:43.557692 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557568    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-oauth-serving-cert\") pod \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\" (UID: \"8aa9b44c-6a8a-4114-9a03-52ee1e053308\") "
Jul 17 18:03:43.557749 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557705    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8aa9b44c-6a8a-4114-9a03-52ee1e053308" (UID: "8aa9b44c-6a8a-4114-9a03-52ee1e053308"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:43.557912 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557798    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-service-ca" (OuterVolumeSpecName: "service-ca") pod "8aa9b44c-6a8a-4114-9a03-52ee1e053308" (UID: "8aa9b44c-6a8a-4114-9a03-52ee1e053308"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:43.557912 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557879    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-config" (OuterVolumeSpecName: "console-config") pod "8aa9b44c-6a8a-4114-9a03-52ee1e053308" (UID: "8aa9b44c-6a8a-4114-9a03-52ee1e053308"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:43.558088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557955    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f696906f-eaaa-4780-b1ed-a0341ecdefc7-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.558088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557974    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2vkt\" (UniqueName: \"kubernetes.io/projected/551208cf-59c7-4400-8473-e3438ef0d16a-kube-api-access-g2vkt\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.558088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557970    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8aa9b44c-6a8a-4114-9a03-52ee1e053308" (UID: "8aa9b44c-6a8a-4114-9a03-52ee1e053308"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:03:43.558088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.557989    2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.558088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.558003    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hgz5\" (UniqueName: \"kubernetes.io/projected/f696906f-eaaa-4780-b1ed-a0341ecdefc7-kube-api-access-5hgz5\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.558088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.558016    2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-trusted-ca-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.559520 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.559498    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8aa9b44c-6a8a-4114-9a03-52ee1e053308" (UID: "8aa9b44c-6a8a-4114-9a03-52ee1e053308"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:03:43.559625 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.559558    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8aa9b44c-6a8a-4114-9a03-52ee1e053308" (UID: "8aa9b44c-6a8a-4114-9a03-52ee1e053308"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:03:43.559625 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.559585    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa9b44c-6a8a-4114-9a03-52ee1e053308-kube-api-access-wk4xv" (OuterVolumeSpecName: "kube-api-access-wk4xv") pod "8aa9b44c-6a8a-4114-9a03-52ee1e053308" (UID: "8aa9b44c-6a8a-4114-9a03-52ee1e053308"). InnerVolumeSpecName "kube-api-access-wk4xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:03:43.590787 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.590756    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"]
Jul 17 18:03:43.593098 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:43.593060    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8295f3fd_3c4f_4cdd_8685_087c250085b4.slice/crio-0d514b325f30b70266ec6eef1ce366244c4ea1b81eeb61f698698b4e33076859 WatchSource:0}: Error finding container 0d514b325f30b70266ec6eef1ce366244c4ea1b81eeb61f698698b4e33076859: Status 404 returned error can't find the container with id 0d514b325f30b70266ec6eef1ce366244c4ea1b81eeb61f698698b4e33076859
Jul 17 18:03:43.594281 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.594228    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"]
Jul 17 18:03:43.597802 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:43.597776    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057591ea_b18e_43ce_897c_371e1a89be63.slice/crio-ddd04123609df41cd144e5d8c6619088ffa5b5c8cf68a0ab9ec60cd09a9afb50 WatchSource:0}: Error finding container ddd04123609df41cd144e5d8c6619088ffa5b5c8cf68a0ab9ec60cd09a9afb50: Status 404 returned error can't find the container with id ddd04123609df41cd144e5d8c6619088ffa5b5c8cf68a0ab9ec60cd09a9afb50
Jul 17 18:03:43.658992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.658960    2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-oauth-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.658992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.658992    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk4xv\" (UniqueName: \"kubernetes.io/projected/8aa9b44c-6a8a-4114-9a03-52ee1e053308-kube-api-access-wk4xv\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.659160 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.659002    2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-oauth-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.659160 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.659011    2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:43.659160 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:43.659020    2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8aa9b44c-6a8a-4114-9a03-52ee1e053308-console-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:44.156011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.155976    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk" event={"ID":"fc78a7d7-9035-4cd4-b12e-56642e234855","Type":"ContainerStarted","Data":"e802bcca31c23368da2b1765c839847cf0b2669f6b7bccbc563cc7edb053c383"}
Jul 17 18:03:44.157481 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.157458    2555 generic.go:356] "Generic (PLEG): container finished" podID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" containerID="67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b" exitCode=0
Jul 17 18:03:44.157596 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.157498    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:44.157596 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.157552    2555 scope.go:117] "RemoveContainer" containerID="67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b"
Jul 17 18:03:44.158962 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:44.158599    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-krqfd\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:44.160080 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.159694    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx" event={"ID":"057591ea-b18e-43ce-897c-371e1a89be63","Type":"ContainerStarted","Data":"2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc"}
Jul 17 18:03:44.160080 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.159749    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:44.160080 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.159777    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx" event={"ID":"057591ea-b18e-43ce-897c-371e1a89be63","Type":"ContainerStarted","Data":"ddd04123609df41cd144e5d8c6619088ffa5b5c8cf68a0ab9ec60cd09a9afb50"}
Jul 17 18:03:44.161264 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.161241    2555 generic.go:356] "Generic (PLEG): container finished" podID="8aa9b44c-6a8a-4114-9a03-52ee1e053308" containerID="3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4" exitCode=0
Jul 17 18:03:44.161352 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.161280    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846f4ff7b6-m6v4l"
Jul 17 18:03:44.161412 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.161279    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846f4ff7b6-m6v4l" event={"ID":"8aa9b44c-6a8a-4114-9a03-52ee1e053308","Type":"ContainerDied","Data":"3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4"}
Jul 17 18:03:44.161412 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.161397    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846f4ff7b6-m6v4l" event={"ID":"8aa9b44c-6a8a-4114-9a03-52ee1e053308","Type":"ContainerDied","Data":"c205b554ee742bbac2b235899753dcc198147a0318723ff6334bbc6d7a84f875"}
Jul 17 18:03:44.162937 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.162918    2555 generic.go:356] "Generic (PLEG): container finished" podID="551208cf-59c7-4400-8473-e3438ef0d16a" containerID="5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826" exitCode=0
Jul 17 18:03:44.163053 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.162963    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:44.165179 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.165146    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk" event={"ID":"8295f3fd-3c4f-4cdd-8685-087c250085b4","Type":"ContainerStarted","Data":"14cffd813b55b1fba82cc6502280bcbbbb3b2709aaccc6602b71280631c00a2d"}
Jul 17 18:03:44.165280 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.165191    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk" event={"ID":"8295f3fd-3c4f-4cdd-8685-087c250085b4","Type":"ContainerStarted","Data":"0d514b325f30b70266ec6eef1ce366244c4ea1b81eeb61f698698b4e33076859"}
Jul 17 18:03:44.165417 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.165402    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"
Jul 17 18:03:44.175403 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.175388    2555 scope.go:117] "RemoveContainer" containerID="67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b"
Jul 17 18:03:44.175721 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:44.175704    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b\": container with ID starting with 67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b not found: ID does not exist" containerID="67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b"
Jul 17 18:03:44.175767 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.175731    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b"} err="failed to get container status \"67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b\": rpc error: code = NotFound desc = could not find container \"67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b\": container with ID starting with 67159c2e763a444bcbd0e62fe6c5d6b5cada8fe2b2f2dd82360177af4463fb6b not found: ID does not exist"
Jul 17 18:03:44.175767 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.175748    2555 scope.go:117] "RemoveContainer" containerID="3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4"
Jul 17 18:03:44.180010 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:44.179984    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:44.180409 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.180371    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xnbfk" podStartSLOduration=1.753937914 podStartE2EDuration="24.180353007s" totalImagesPullingTime="22.426415093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:44.17761618 +0000 UTC m=+421.763092443" watchObservedRunningTime="2026-07-17 18:03:44.180353007 +0000 UTC m=+421.765829259"
Jul 17 18:03:44.185769 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.185730    2555 scope.go:117] "RemoveContainer" containerID="3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4"
Jul 17 18:03:44.186111 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:44.186091    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4\": container with ID starting with 3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4 not found: ID does not exist" containerID="3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4"
Jul 17 18:03:44.186212 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.186119    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4"} err="failed to get container status \"3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4\": rpc error: code = NotFound desc = could not find container \"3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4\": container with ID starting with 3107f39dd3b520e4731c069b655ab91ea5c81d2b3ae1c6a1c58206ccb43167b4 not found: ID does not exist"
Jul 17 18:03:44.186212 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.186143    2555 scope.go:117] "RemoveContainer" containerID="5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826"
Jul 17 18:03:44.196132 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.196110    2555 scope.go:117] "RemoveContainer" containerID="5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826"
Jul 17 18:03:44.196425 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:44.196406    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826\": container with ID starting with 5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826 not found: ID does not exist" containerID="5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826"
Jul 17 18:03:44.196497 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.196433    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826"} err="failed to get container status \"5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826\": rpc error: code = NotFound desc = could not find container \"5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826\": container with ID starting with 5885b7a53e583977876e2514165abdd8148c5e00f9ac2da7c4d717a2776ff826 not found: ID does not exist"
Jul 17 18:03:44.204021 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:44.203991    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-krqfd\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krqfd"
Jul 17 18:03:44.204652 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.204593    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk" podStartSLOduration=14.204575873 podStartE2EDuration="14.204575873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:44.201853685 +0000 UTC m=+421.787329937" watchObservedRunningTime="2026-07-17 18:03:44.204575873 +0000 UTC m=+421.790052126"
Jul 17 18:03:44.206045 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:44.206023    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pvvph\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pvvph"
Jul 17 18:03:44.226309 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.226251    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx" podStartSLOduration=14.226231844 podStartE2EDuration="14.226231844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:44.224453719 +0000 UTC m=+421.809929973" watchObservedRunningTime="2026-07-17 18:03:44.226231844 +0000 UTC m=+421.811708097"
Jul 17 18:03:44.238450 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.238419    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846f4ff7b6-m6v4l"]
Jul 17 18:03:44.242383 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.242358    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-846f4ff7b6-m6v4l"]
Jul 17 18:03:44.944085 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.944051    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="551208cf-59c7-4400-8473-e3438ef0d16a" path="/var/lib/kubelet/pods/551208cf-59c7-4400-8473-e3438ef0d16a/volumes"
Jul 17 18:03:44.944430 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.944416    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aa9b44c-6a8a-4114-9a03-52ee1e053308" path="/var/lib/kubelet/pods/8aa9b44c-6a8a-4114-9a03-52ee1e053308/volumes"
Jul 17 18:03:44.944785 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:44.944772    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f696906f-eaaa-4780-b1ed-a0341ecdefc7" path="/var/lib/kubelet/pods/f696906f-eaaa-4780-b1ed-a0341ecdefc7/volumes"
Jul 17 18:03:45.732272 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:45.732114    2555 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-m4wkt_kuadrant-system_e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87_0(a75220e0e5b6cf28728c6972eb3c470a5ce26c0fcee1b8c7ed6296899cc62fe7): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-m4wkt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 17 18:03:45.732272 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:45.732180    2555 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-m4wkt_kuadrant-system_e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87_0(a75220e0e5b6cf28728c6972eb3c470a5ce26c0fcee1b8c7ed6296899cc62fe7): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-m4wkt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:46.180257 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.180223    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:46.182529 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:46.182503    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:46.188885 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.188863    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:46.191021 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:46.190995    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:46.287477 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.287437    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdm5f\" (UniqueName: \"kubernetes.io/projected/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-kube-api-access-fdm5f\") pod \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") "
Jul 17 18:03:46.287477 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.287474    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-extensions-socket-volume\") pod \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\" (UID: \"e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87\") "
Jul 17 18:03:46.288037 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.287994    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87" (UID: "e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:03:46.288147 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.288119    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:46.289831 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.289807    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-kube-api-access-fdm5f" (OuterVolumeSpecName: "kube-api-access-fdm5f") pod "e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87" (UID: "e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87"). InnerVolumeSpecName "kube-api-access-fdm5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:03:46.388850 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.388809    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdm5f\" (UniqueName: \"kubernetes.io/projected/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87-kube-api-access-fdm5f\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:03:46.945044 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:46.945011    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87" path="/var/lib/kubelet/pods/e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87/volumes"
Jul 17 18:03:47.186659 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:47.186612    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:47.188975 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:47.188937    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:47.190748 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:03:47.190721    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m4wkt\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="e2fbf9bf-0ddb-4f89-8e44-8f654b3e0e87" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m4wkt"
Jul 17 18:03:55.176483 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.176451    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rjpk"
Jul 17 18:03:55.176917 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.176520    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:03:55.654326 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.654288    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"]
Jul 17 18:03:55.654707 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.654695    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa9b44c-6a8a-4114-9a03-52ee1e053308" containerName="console"
Jul 17 18:03:55.654771 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.654709    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa9b44c-6a8a-4114-9a03-52ee1e053308" containerName="console"
Jul 17 18:03:55.654811 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.654790    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa9b44c-6a8a-4114-9a03-52ee1e053308" containerName="console"
Jul 17 18:03:55.660860 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.660840    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:55.668473 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.668446    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfdxn\" (UniqueName: \"kubernetes.io/projected/055b1caa-1462-4d2f-8848-c4e65728f6f5-kube-api-access-nfdxn\") pod \"kuadrant-operator-controller-manager-86878c769f-7rrtv\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:55.668589 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.668573    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/055b1caa-1462-4d2f-8848-c4e65728f6f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7rrtv\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:55.674819 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.674796    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"]
Jul 17 18:03:55.769115 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.769082    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/055b1caa-1462-4d2f-8848-c4e65728f6f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7rrtv\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:55.769307 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.769149    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfdxn\" (UniqueName: \"kubernetes.io/projected/055b1caa-1462-4d2f-8848-c4e65728f6f5-kube-api-access-nfdxn\") pod \"kuadrant-operator-controller-manager-86878c769f-7rrtv\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:55.769459 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.769440    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/055b1caa-1462-4d2f-8848-c4e65728f6f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7rrtv\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:55.789141 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.789105    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfdxn\" (UniqueName: \"kubernetes.io/projected/055b1caa-1462-4d2f-8848-c4e65728f6f5-kube-api-access-nfdxn\") pod \"kuadrant-operator-controller-manager-86878c769f-7rrtv\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:55.971109 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:55.971061    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:56.156268 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:56.156232    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"]
Jul 17 18:03:56.159963 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:03:56.159936    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055b1caa_1462_4d2f_8848_c4e65728f6f5.slice/crio-d88cb9799f208efe86d3b01a9847a3af3cf43bea161a5774e1dccb13afe9ad20 WatchSource:0}: Error finding container d88cb9799f208efe86d3b01a9847a3af3cf43bea161a5774e1dccb13afe9ad20: Status 404 returned error can't find the container with id d88cb9799f208efe86d3b01a9847a3af3cf43bea161a5774e1dccb13afe9ad20
Jul 17 18:03:56.238361 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:56.238331    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv" event={"ID":"055b1caa-1462-4d2f-8848-c4e65728f6f5","Type":"ContainerStarted","Data":"08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e"}
Jul 17 18:03:56.238761 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:56.238409    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv" event={"ID":"055b1caa-1462-4d2f-8848-c4e65728f6f5","Type":"ContainerStarted","Data":"d88cb9799f208efe86d3b01a9847a3af3cf43bea161a5774e1dccb13afe9ad20"}
Jul 17 18:03:56.238761 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:56.238565    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:03:56.263688 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:03:56.263608    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv" podStartSLOduration=1.263593029 podStartE2EDuration="1.263593029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:03:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:03:56.260551018 +0000 UTC m=+433.846027281" watchObservedRunningTime="2026-07-17 18:03:56.263593029 +0000 UTC m=+433.849069281"
Jul 17 18:04:07.245899 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.245818    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:04:07.295266 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.295230    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"]
Jul 17 18:04:07.295528 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.295498    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx" podUID="057591ea-b18e-43ce-897c-371e1a89be63" containerName="manager" containerID="cri-o://2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc" gracePeriod=10
Jul 17 18:04:07.695414 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.695369    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:04:07.791096 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.791008    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057591ea-b18e-43ce-897c-371e1a89be63-extensions-socket-volume\") pod \"057591ea-b18e-43ce-897c-371e1a89be63\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") "
Jul 17 18:04:07.791096 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.791048    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktsjv\" (UniqueName: \"kubernetes.io/projected/057591ea-b18e-43ce-897c-371e1a89be63-kube-api-access-ktsjv\") pod \"057591ea-b18e-43ce-897c-371e1a89be63\" (UID: \"057591ea-b18e-43ce-897c-371e1a89be63\") "
Jul 17 18:04:07.791396 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.791368    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/057591ea-b18e-43ce-897c-371e1a89be63-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "057591ea-b18e-43ce-897c-371e1a89be63" (UID: "057591ea-b18e-43ce-897c-371e1a89be63"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:04:07.793202 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.793176    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/057591ea-b18e-43ce-897c-371e1a89be63-kube-api-access-ktsjv" (OuterVolumeSpecName: "kube-api-access-ktsjv") pod "057591ea-b18e-43ce-897c-371e1a89be63" (UID: "057591ea-b18e-43ce-897c-371e1a89be63"). InnerVolumeSpecName "kube-api-access-ktsjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:04:07.891992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.891953    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057591ea-b18e-43ce-897c-371e1a89be63-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:04:07.891992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:07.891983    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktsjv\" (UniqueName: \"kubernetes.io/projected/057591ea-b18e-43ce-897c-371e1a89be63-kube-api-access-ktsjv\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:04:08.307967 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.307934    2555 generic.go:356] "Generic (PLEG): container finished" podID="057591ea-b18e-43ce-897c-371e1a89be63" containerID="2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc" exitCode=0
Jul 17 18:04:08.308384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.307976    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"
Jul 17 18:04:08.308384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.308015    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx" event={"ID":"057591ea-b18e-43ce-897c-371e1a89be63","Type":"ContainerDied","Data":"2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc"}
Jul 17 18:04:08.308384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.308066    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx" event={"ID":"057591ea-b18e-43ce-897c-371e1a89be63","Type":"ContainerDied","Data":"ddd04123609df41cd144e5d8c6619088ffa5b5c8cf68a0ab9ec60cd09a9afb50"}
Jul 17 18:04:08.308384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.308087    2555 scope.go:117] "RemoveContainer" containerID="2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc"
Jul 17 18:04:08.318288 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.318262    2555 scope.go:117] "RemoveContainer" containerID="2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc"
Jul 17 18:04:08.318589 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:04:08.318571    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc\": container with ID starting with 2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc not found: ID does not exist" containerID="2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc"
Jul 17 18:04:08.318690 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.318602    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc"} err="failed to get container status \"2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc\": rpc error: code = NotFound desc = could not find container \"2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc\": container with ID starting with 2b3fde685a0158fde69c28e3fd4421f5175bf218a3bebf51999e2de8f7f911cc not found: ID does not exist"
Jul 17 18:04:08.333897 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.333861    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"]
Jul 17 18:04:08.349220 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.349192    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pnvdx"]
Jul 17 18:04:08.945770 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:08.945730    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="057591ea-b18e-43ce-897c-371e1a89be63" path="/var/lib/kubelet/pods/057591ea-b18e-43ce-897c-371e1a89be63/volumes"
Jul 17 18:04:14.747160 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.747126    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:04:14.747620 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.747605    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057591ea-b18e-43ce-897c-371e1a89be63" containerName="manager"
Jul 17 18:04:14.747689 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.747624    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="057591ea-b18e-43ce-897c-371e1a89be63" containerName="manager"
Jul 17 18:04:14.747752 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.747742    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="057591ea-b18e-43ce-897c-371e1a89be63" containerName="manager"
Jul 17 18:04:14.830218 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.830175    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:04:14.830218 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.830215    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:04:14.830420 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.830312    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:14.832779 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.832753    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 18:04:14.956114 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.956068    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72209bda-6da3-4cd6-a0f0-6597c8628036-config-file\") pod \"limitador-limitador-54d6d988d5-cbfck\" (UID: \"72209bda-6da3-4cd6-a0f0-6597c8628036\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:14.956293 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:14.956134    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s7bh\" (UniqueName: \"kubernetes.io/projected/72209bda-6da3-4cd6-a0f0-6597c8628036-kube-api-access-4s7bh\") pod \"limitador-limitador-54d6d988d5-cbfck\" (UID: \"72209bda-6da3-4cd6-a0f0-6597c8628036\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:15.057271 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.057180    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72209bda-6da3-4cd6-a0f0-6597c8628036-config-file\") pod \"limitador-limitador-54d6d988d5-cbfck\" (UID: \"72209bda-6da3-4cd6-a0f0-6597c8628036\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:15.057271 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.057240    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s7bh\" (UniqueName: \"kubernetes.io/projected/72209bda-6da3-4cd6-a0f0-6597c8628036-kube-api-access-4s7bh\") pod \"limitador-limitador-54d6d988d5-cbfck\" (UID: \"72209bda-6da3-4cd6-a0f0-6597c8628036\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:15.057875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.057854    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/72209bda-6da3-4cd6-a0f0-6597c8628036-config-file\") pod \"limitador-limitador-54d6d988d5-cbfck\" (UID: \"72209bda-6da3-4cd6-a0f0-6597c8628036\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:15.065451 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.065417    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s7bh\" (UniqueName: \"kubernetes.io/projected/72209bda-6da3-4cd6-a0f0-6597c8628036-kube-api-access-4s7bh\") pod \"limitador-limitador-54d6d988d5-cbfck\" (UID: \"72209bda-6da3-4cd6-a0f0-6597c8628036\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:15.140759 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.140717    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:15.165907 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.165872    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-v5zfc"]
Jul 17 18:04:15.169776 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.169688    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v5zfc"
Jul 17 18:04:15.172177 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.172151    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zxtj5\""
Jul 17 18:04:15.175704 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.175675    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-v5zfc"]
Jul 17 18:04:15.259368 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.259332    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbwgp\" (UniqueName: \"kubernetes.io/projected/9f6ae241-b719-40b3-9f24-d52e0b8873e6-kube-api-access-fbwgp\") pod \"authorino-58cd955675-v5zfc\" (UID: \"9f6ae241-b719-40b3-9f24-d52e0b8873e6\") " pod="kuadrant-system/authorino-58cd955675-v5zfc"
Jul 17 18:04:15.290220 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.290044    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:04:15.293099 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:04:15.293068    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72209bda_6da3_4cd6_a0f0_6597c8628036.slice/crio-062d5f612d7b56746e4fd77dede8f1ca9522b2adf34577ef905b90a6959e4b29 WatchSource:0}: Error finding container 062d5f612d7b56746e4fd77dede8f1ca9522b2adf34577ef905b90a6959e4b29: Status 404 returned error can't find the container with id 062d5f612d7b56746e4fd77dede8f1ca9522b2adf34577ef905b90a6959e4b29
Jul 17 18:04:15.352921 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.352833    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck" event={"ID":"72209bda-6da3-4cd6-a0f0-6597c8628036","Type":"ContainerStarted","Data":"062d5f612d7b56746e4fd77dede8f1ca9522b2adf34577ef905b90a6959e4b29"}
Jul 17 18:04:15.360240 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.360202    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwgp\" (UniqueName: \"kubernetes.io/projected/9f6ae241-b719-40b3-9f24-d52e0b8873e6-kube-api-access-fbwgp\") pod \"authorino-58cd955675-v5zfc\" (UID: \"9f6ae241-b719-40b3-9f24-d52e0b8873e6\") " pod="kuadrant-system/authorino-58cd955675-v5zfc"
Jul 17 18:04:15.370342 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.370314    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbwgp\" (UniqueName: \"kubernetes.io/projected/9f6ae241-b719-40b3-9f24-d52e0b8873e6-kube-api-access-fbwgp\") pod \"authorino-58cd955675-v5zfc\" (UID: \"9f6ae241-b719-40b3-9f24-d52e0b8873e6\") " pod="kuadrant-system/authorino-58cd955675-v5zfc"
Jul 17 18:04:15.482253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.482216    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v5zfc"
Jul 17 18:04:15.616115 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:15.616085    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-v5zfc"]
Jul 17 18:04:15.617931 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:04:15.617905    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f6ae241_b719_40b3_9f24_d52e0b8873e6.slice/crio-4a1b3a850c9945b9cde533e5571cb8e1b0f1de81fe38eafc29271800971f1e87 WatchSource:0}: Error finding container 4a1b3a850c9945b9cde533e5571cb8e1b0f1de81fe38eafc29271800971f1e87: Status 404 returned error can't find the container with id 4a1b3a850c9945b9cde533e5571cb8e1b0f1de81fe38eafc29271800971f1e87
Jul 17 18:04:16.365863 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:16.365823    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v5zfc" event={"ID":"9f6ae241-b719-40b3-9f24-d52e0b8873e6","Type":"ContainerStarted","Data":"4a1b3a850c9945b9cde533e5571cb8e1b0f1de81fe38eafc29271800971f1e87"}
Jul 17 18:04:19.754675 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:19.754623    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-v5zfc"]
Jul 17 18:04:20.392200 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.392167    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v5zfc" event={"ID":"9f6ae241-b719-40b3-9f24-d52e0b8873e6","Type":"ContainerStarted","Data":"8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee"}
Jul 17 18:04:20.392200 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.392184    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-v5zfc" podUID="9f6ae241-b719-40b3-9f24-d52e0b8873e6" containerName="authorino" containerID="cri-o://8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee" gracePeriod=30
Jul 17 18:04:20.393909 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.393878    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck" event={"ID":"72209bda-6da3-4cd6-a0f0-6597c8628036","Type":"ContainerStarted","Data":"f3142eb76a131802be4f47cdc41f7483e044ae7550d2f311ed4753e3164d3019"}
Jul 17 18:04:20.394047 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.394019    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:20.407540 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.407496    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-v5zfc" podStartSLOduration=1.320442082 podStartE2EDuration="5.407479698s" totalImagesPullingTime="4.087037616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:04:20.405218183 +0000 UTC m=+457.990694434" watchObservedRunningTime="2026-07-17 18:04:20.407479698 +0000 UTC m=+457.992955950"
Jul 17 18:04:20.421341 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.421294    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck" podStartSLOduration=2.011539199 podStartE2EDuration="6.421278767s" totalImagesPullingTime="4.409739568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:04:20.420710568 +0000 UTC m=+458.006186822" watchObservedRunningTime="2026-07-17 18:04:20.421278767 +0000 UTC m=+458.006755019"
Jul 17 18:04:20.646353 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.646284    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v5zfc"
Jul 17 18:04:20.813232 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.813199    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbwgp\" (UniqueName: \"kubernetes.io/projected/9f6ae241-b719-40b3-9f24-d52e0b8873e6-kube-api-access-fbwgp\") pod \"9f6ae241-b719-40b3-9f24-d52e0b8873e6\" (UID: \"9f6ae241-b719-40b3-9f24-d52e0b8873e6\") "
Jul 17 18:04:20.815292 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.815256    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6ae241-b719-40b3-9f24-d52e0b8873e6-kube-api-access-fbwgp" (OuterVolumeSpecName: "kube-api-access-fbwgp") pod "9f6ae241-b719-40b3-9f24-d52e0b8873e6" (UID: "9f6ae241-b719-40b3-9f24-d52e0b8873e6"). InnerVolumeSpecName "kube-api-access-fbwgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:04:20.913952 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:20.913861    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbwgp\" (UniqueName: \"kubernetes.io/projected/9f6ae241-b719-40b3-9f24-d52e0b8873e6-kube-api-access-fbwgp\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:04:21.401031 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.400996    2555 generic.go:356] "Generic (PLEG): container finished" podID="9f6ae241-b719-40b3-9f24-d52e0b8873e6" containerID="8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee" exitCode=0
Jul 17 18:04:21.401249 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.401039    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v5zfc"
Jul 17 18:04:21.401249 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.401090    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v5zfc" event={"ID":"9f6ae241-b719-40b3-9f24-d52e0b8873e6","Type":"ContainerDied","Data":"8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee"}
Jul 17 18:04:21.401249 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.401127    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v5zfc" event={"ID":"9f6ae241-b719-40b3-9f24-d52e0b8873e6","Type":"ContainerDied","Data":"4a1b3a850c9945b9cde533e5571cb8e1b0f1de81fe38eafc29271800971f1e87"}
Jul 17 18:04:21.401249 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.401146    2555 scope.go:117] "RemoveContainer" containerID="8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee"
Jul 17 18:04:21.410512 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.410486    2555 scope.go:117] "RemoveContainer" containerID="8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee"
Jul 17 18:04:21.410843 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:04:21.410826    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee\": container with ID starting with 8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee not found: ID does not exist" containerID="8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee"
Jul 17 18:04:21.410914 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.410852    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee"} err="failed to get container status \"8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee\": rpc error: code = NotFound desc = could not find container \"8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee\": container with ID starting with 8e519a74cac76382efeedf8ba368faa064c535f004619563b90275587da5d0ee not found: ID does not exist"
Jul 17 18:04:21.417958 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.417930    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-v5zfc"]
Jul 17 18:04:21.421666 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:21.421627    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-v5zfc"]
Jul 17 18:04:22.944252 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:22.944216    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f6ae241-b719-40b3-9f24-d52e0b8873e6" path="/var/lib/kubelet/pods/9f6ae241-b719-40b3-9f24-d52e0b8873e6/volumes"
Jul 17 18:04:31.402702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:31.402674    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cbfck"
Jul 17 18:04:54.511618 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.511585    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c5568d77b-pjznq"]
Jul 17 18:04:54.512046 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.512029    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6ae241-b719-40b3-9f24-d52e0b8873e6" containerName="authorino"
Jul 17 18:04:54.512046 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.512042    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6ae241-b719-40b3-9f24-d52e0b8873e6" containerName="authorino"
Jul 17 18:04:54.512125 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.512117    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6ae241-b719-40b3-9f24-d52e0b8873e6" containerName="authorino"
Jul 17 18:04:54.518946 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.518920    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:54.521453 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.521431    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9dmbt\""
Jul 17 18:04:54.521595 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.521468    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 18:04:54.521595 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.521514    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 18:04:54.525786 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.525762    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c5568d77b-pjznq"]
Jul 17 18:04:54.592044 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.592003    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:54.592044 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.592042    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npr8h\" (UniqueName: \"kubernetes.io/projected/72f16c1c-c9c2-49c1-9e8c-1f411225a401-kube-api-access-npr8h\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:54.693155 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.693123    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:54.693155 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.693156    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npr8h\" (UniqueName: \"kubernetes.io/projected/72f16c1c-c9c2-49c1-9e8c-1f411225a401-kube-api-access-npr8h\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:54.693411 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:04:54.693280    2555 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 17 18:04:54.693411 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:04:54.693345    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert podName:72f16c1c-c9c2-49c1-9e8c-1f411225a401 nodeName:}" failed. No retries permitted until 2026-07-17 18:04:55.193323652 +0000 UTC m=+492.778799902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert") pod "maas-controller-c5568d77b-pjznq" (UID: "72f16c1c-c9c2-49c1-9e8c-1f411225a401") : secret "maas-controller-webhook-cert" not found
Jul 17 18:04:54.703941 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:54.703904    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npr8h\" (UniqueName: \"kubernetes.io/projected/72f16c1c-c9c2-49c1-9e8c-1f411225a401-kube-api-access-npr8h\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:55.197868 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:55.197826    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:55.198048 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:04:55.197985    2555 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 17 18:04:55.198189 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:04:55.198049    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert podName:72f16c1c-c9c2-49c1-9e8c-1f411225a401 nodeName:}" failed. No retries permitted until 2026-07-17 18:04:56.198030358 +0000 UTC m=+493.783506592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert") pod "maas-controller-c5568d77b-pjznq" (UID: "72f16c1c-c9c2-49c1-9e8c-1f411225a401") : secret "maas-controller-webhook-cert" not found
Jul 17 18:04:56.206173 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:56.206130    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:56.208669 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:56.208626    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert\") pod \"maas-controller-c5568d77b-pjznq\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") " pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:56.330550 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:56.330508    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:56.469235 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:56.469207    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c5568d77b-pjznq"]
Jul 17 18:04:56.471220 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:04:56.471186    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f16c1c_c9c2_49c1_9e8c_1f411225a401.slice/crio-0b572a7eda3d02f14a941322d91697823c91ee49b5645d9db31ccf569c266f2b WatchSource:0}: Error finding container 0b572a7eda3d02f14a941322d91697823c91ee49b5645d9db31ccf569c266f2b: Status 404 returned error can't find the container with id 0b572a7eda3d02f14a941322d91697823c91ee49b5645d9db31ccf569c266f2b
Jul 17 18:04:56.612773 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:56.612737    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c5568d77b-pjznq" event={"ID":"72f16c1c-c9c2-49c1-9e8c-1f411225a401","Type":"ContainerStarted","Data":"0b572a7eda3d02f14a941322d91697823c91ee49b5645d9db31ccf569c266f2b"}
Jul 17 18:04:59.634137 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:59.634093    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c5568d77b-pjznq" event={"ID":"72f16c1c-c9c2-49c1-9e8c-1f411225a401","Type":"ContainerStarted","Data":"229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e"}
Jul 17 18:04:59.634548 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:59.634172    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:04:59.652243 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:04:59.652188    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c5568d77b-pjznq" podStartSLOduration=3.357856864 podStartE2EDuration="5.652168025s" totalImagesPullingTime="2.294311161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:04:59.651093803 +0000 UTC m=+497.236570056" watchObservedRunningTime="2026-07-17 18:04:59.652168025 +0000 UTC m=+497.237644274"
Jul 17 18:05:09.971223 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:09.971189    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c5568d77b-pjznq"]
Jul 17 18:05:09.971622 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:09.971386    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c5568d77b-pjznq" podUID="72f16c1c-c9c2-49c1-9e8c-1f411225a401" containerName="manager" containerID="cri-o://229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e" gracePeriod=30
Jul 17 18:05:09.973071 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:09.973047    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:05:10.222451 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.222387    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:05:10.327218 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.327187    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert\") pod \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") "
Jul 17 18:05:10.327415 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.327240    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npr8h\" (UniqueName: \"kubernetes.io/projected/72f16c1c-c9c2-49c1-9e8c-1f411225a401-kube-api-access-npr8h\") pod \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\" (UID: \"72f16c1c-c9c2-49c1-9e8c-1f411225a401\") "
Jul 17 18:05:10.329543 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.329518    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "72f16c1c-c9c2-49c1-9e8c-1f411225a401" (UID: "72f16c1c-c9c2-49c1-9e8c-1f411225a401"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:05:10.329684 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.329517    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f16c1c-c9c2-49c1-9e8c-1f411225a401-kube-api-access-npr8h" (OuterVolumeSpecName: "kube-api-access-npr8h") pod "72f16c1c-c9c2-49c1-9e8c-1f411225a401" (UID: "72f16c1c-c9c2-49c1-9e8c-1f411225a401"). InnerVolumeSpecName "kube-api-access-npr8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:05:10.428586 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.428558    2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f16c1c-c9c2-49c1-9e8c-1f411225a401-webhook-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:05:10.428586 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.428582    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npr8h\" (UniqueName: \"kubernetes.io/projected/72f16c1c-c9c2-49c1-9e8c-1f411225a401-kube-api-access-npr8h\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:05:10.696242 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.696206    2555 generic.go:356] "Generic (PLEG): container finished" podID="72f16c1c-c9c2-49c1-9e8c-1f411225a401" containerID="229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e" exitCode=0
Jul 17 18:05:10.696424 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.696254    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c5568d77b-pjznq"
Jul 17 18:05:10.696424 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.696275    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c5568d77b-pjznq" event={"ID":"72f16c1c-c9c2-49c1-9e8c-1f411225a401","Type":"ContainerDied","Data":"229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e"}
Jul 17 18:05:10.696424 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.696317    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c5568d77b-pjznq" event={"ID":"72f16c1c-c9c2-49c1-9e8c-1f411225a401","Type":"ContainerDied","Data":"0b572a7eda3d02f14a941322d91697823c91ee49b5645d9db31ccf569c266f2b"}
Jul 17 18:05:10.696424 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.696338    2555 scope.go:117] "RemoveContainer" containerID="229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e"
Jul 17 18:05:10.708393 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.708376    2555 scope.go:117] "RemoveContainer" containerID="229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e"
Jul 17 18:05:10.708705 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:05:10.708687    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e\": container with ID starting with 229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e not found: ID does not exist" containerID="229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e"
Jul 17 18:05:10.708764 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.708715    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e"} err="failed to get container status \"229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e\": rpc error: code = NotFound desc = could not find container \"229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e\": container with ID starting with 229e30fd19da3c925a6757a53c7918bdb979c9fe1976622d3b7ded765d6d2b4e not found: ID does not exist"
Jul 17 18:05:10.723618 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.723587    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c5568d77b-pjznq"]
Jul 17 18:05:10.729704 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.729676    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c5568d77b-pjznq"]
Jul 17 18:05:10.943412 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:10.943376    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f16c1c-c9c2-49c1-9e8c-1f411225a401" path="/var/lib/kubelet/pods/72f16c1c-c9c2-49c1-9e8c-1f411225a401/volumes"
Jul 17 18:05:31.629725 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.629687    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"]
Jul 17 18:05:31.630177 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.630067    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f16c1c-c9c2-49c1-9e8c-1f411225a401" containerName="manager"
Jul 17 18:05:31.630177 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.630078    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f16c1c-c9c2-49c1-9e8c-1f411225a401" containerName="manager"
Jul 17 18:05:31.630177 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.630163    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f16c1c-c9c2-49c1-9e8c-1f411225a401" containerName="manager"
Jul 17 18:05:31.634940 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.634911    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:31.637839 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.637816    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 17 18:05:31.644154 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.644118    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"]
Jul 17 18:05:31.695964 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.695930    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:05:31.725940 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.725911    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ace2039-18db-4d7b-90b0-303fbf522b6c-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zkwv7\" (UID: \"3ace2039-18db-4d7b-90b0-303fbf522b6c\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:31.726110 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.725950    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjtq\" (UniqueName: \"kubernetes.io/projected/3ace2039-18db-4d7b-90b0-303fbf522b6c-kube-api-access-pkjtq\") pod \"payload-pre-processing-7677b774bb-zkwv7\" (UID: \"3ace2039-18db-4d7b-90b0-303fbf522b6c\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:31.827167 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.827130    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ace2039-18db-4d7b-90b0-303fbf522b6c-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zkwv7\" (UID: \"3ace2039-18db-4d7b-90b0-303fbf522b6c\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:31.827167 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.827173    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjtq\" (UniqueName: \"kubernetes.io/projected/3ace2039-18db-4d7b-90b0-303fbf522b6c-kube-api-access-pkjtq\") pod \"payload-pre-processing-7677b774bb-zkwv7\" (UID: \"3ace2039-18db-4d7b-90b0-303fbf522b6c\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:31.827894 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.827870    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ace2039-18db-4d7b-90b0-303fbf522b6c-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zkwv7\" (UID: \"3ace2039-18db-4d7b-90b0-303fbf522b6c\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:31.834813 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.834786    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjtq\" (UniqueName: \"kubernetes.io/projected/3ace2039-18db-4d7b-90b0-303fbf522b6c-kube-api-access-pkjtq\") pod \"payload-pre-processing-7677b774bb-zkwv7\" (UID: \"3ace2039-18db-4d7b-90b0-303fbf522b6c\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:31.947542 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:31.947446    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:32.090018 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:32.089985    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"]
Jul 17 18:05:32.093515 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:05:32.093484    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ace2039_18db_4d7b_90b0_303fbf522b6c.slice/crio-cf6a74d5745d8673b787c64650d6cea137c09adebe004e97c5ba3c784c26ee41 WatchSource:0}: Error finding container cf6a74d5745d8673b787c64650d6cea137c09adebe004e97c5ba3c784c26ee41: Status 404 returned error can't find the container with id cf6a74d5745d8673b787c64650d6cea137c09adebe004e97c5ba3c784c26ee41
Jul 17 18:05:32.823436 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:32.823389    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7" event={"ID":"3ace2039-18db-4d7b-90b0-303fbf522b6c","Type":"ContainerStarted","Data":"cf6a74d5745d8673b787c64650d6cea137c09adebe004e97c5ba3c784c26ee41"}
Jul 17 18:05:33.830241 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:33.830205    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7" event={"ID":"3ace2039-18db-4d7b-90b0-303fbf522b6c","Type":"ContainerStarted","Data":"d7548aa0639152285cc42e06a9625509de1ca18e377685ca9f05eb744fecc7d4"}
Jul 17 18:05:33.830720 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:33.830301    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:33.847195 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:33.847136    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7" podStartSLOduration=1.210013801 podStartE2EDuration="2.847118837s" totalImagesPullingTime="1.637105036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:05:33.845192944 +0000 UTC m=+531.430669200" watchObservedRunningTime="2026-07-17 18:05:33.847118837 +0000 UTC m=+531.432595089"
Jul 17 18:05:44.839203 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:44.839170    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zkwv7"
Jul 17 18:05:57.889224 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:05:57.889181    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:06:07.691606 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.691566    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"]
Jul 17 18:06:07.694207 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.694190    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.696869 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.696846    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 17 18:06:07.697812 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.697792    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gsgfc\""
Jul 17 18:06:07.697936 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.697809    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 18:06:07.697936 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.697811    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 18:06:07.704440 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.704416    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"]
Jul 17 18:06:07.845844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.845798    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.846024 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.845850    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqm9s\" (UniqueName: \"kubernetes.io/projected/577243a4-ad30-408e-9e50-348b806771f2-kube-api-access-qqm9s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.846024 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.845894    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577243a4-ad30-408e-9e50-348b806771f2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.846024 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.845921    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.846171 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.846046    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.846171 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.846100    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.946917 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.946823    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.946917 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.946885    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.947159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.946942    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.947159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.946966    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqm9s\" (UniqueName: \"kubernetes.io/projected/577243a4-ad30-408e-9e50-348b806771f2-kube-api-access-qqm9s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.947159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.946998    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577243a4-ad30-408e-9e50-348b806771f2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.947159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.947022    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.947371 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.947264    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.947371 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.947351    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.947452 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.947428    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.949230 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.949208    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577243a4-ad30-408e-9e50-348b806771f2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.949487 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.949469    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577243a4-ad30-408e-9e50-348b806771f2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:07.954535 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:07.954510    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqm9s\" (UniqueName: \"kubernetes.io/projected/577243a4-ad30-408e-9e50-348b806771f2-kube-api-access-qqm9s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zth9g\" (UID: \"577243a4-ad30-408e-9e50-348b806771f2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:08.006039 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:08.005996    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:08.153686 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:08.153657    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"]
Jul 17 18:06:08.155528 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:06:08.155504    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577243a4_ad30_408e_9e50_348b806771f2.slice/crio-c526131533173ee4078b8dda946ce6101bbb37147a99b84683d17b751845c23d WatchSource:0}: Error finding container c526131533173ee4078b8dda946ce6101bbb37147a99b84683d17b751845c23d: Status 404 returned error can't find the container with id c526131533173ee4078b8dda946ce6101bbb37147a99b84683d17b751845c23d
Jul 17 18:06:08.599675 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:08.599586    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:06:09.030398 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:09.030363    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g" event={"ID":"577243a4-ad30-408e-9e50-348b806771f2","Type":"ContainerStarted","Data":"c526131533173ee4078b8dda946ce6101bbb37147a99b84683d17b751845c23d"}
Jul 17 18:06:10.039186 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:10.039133    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g" event={"ID":"577243a4-ad30-408e-9e50-348b806771f2","Type":"ContainerStarted","Data":"1622da184c649df789883effdc466ab2b759beb22d38ed8602344c39a8b3266e"}
Jul 17 18:06:10.059544 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:10.059479    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g" podStartSLOduration=2.084663088 podStartE2EDuration="3.059453242s" totalImagesPullingTime="974.790154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:06:10.055923387 +0000 UTC m=+567.641399640" watchObservedRunningTime="2026-07-17 18:06:10.059453242 +0000 UTC m=+567.644929494"
Jul 17 18:06:14.091534 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.091321    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"]
Jul 17 18:06:14.093948 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.093926    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.096408 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.096375    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 17 18:06:14.107401 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.107376    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"]
Jul 17 18:06:14.207087 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.207040    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.207087 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.207084    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5484266e-fb83-4cf4-99d2-46c331f78aba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.207333 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.207166    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.207333 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.207218    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.207333 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.207241    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.207333 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.207274    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx7cw\" (UniqueName: \"kubernetes.io/projected/5484266e-fb83-4cf4-99d2-46c331f78aba-kube-api-access-tx7cw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.307749 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.307712    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.307749 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.307749    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.308019 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.307779    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx7cw\" (UniqueName: \"kubernetes.io/projected/5484266e-fb83-4cf4-99d2-46c331f78aba-kube-api-access-tx7cw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.308019 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.307807    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.308019 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.307826    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5484266e-fb83-4cf4-99d2-46c331f78aba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.308019 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.307884    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.308233 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.308163    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.308278 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.308230    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.308313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.308269    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.310099 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.310076    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5484266e-fb83-4cf4-99d2-46c331f78aba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.310827 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.310761    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5484266e-fb83-4cf4-99d2-46c331f78aba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.315710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.315678    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx7cw\" (UniqueName: \"kubernetes.io/projected/5484266e-fb83-4cf4-99d2-46c331f78aba-kube-api-access-tx7cw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-46n6s\" (UID: \"5484266e-fb83-4cf4-99d2-46c331f78aba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.405386 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.405288    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:14.551058 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:14.551030    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"]
Jul 17 18:06:14.553032 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:06:14.553003    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5484266e_fb83_4cf4_99d2_46c331f78aba.slice/crio-b900d176cd1b5ab0a117884dbf763ac0e66179c060616beb175c7935681ba339 WatchSource:0}: Error finding container b900d176cd1b5ab0a117884dbf763ac0e66179c060616beb175c7935681ba339: Status 404 returned error can't find the container with id b900d176cd1b5ab0a117884dbf763ac0e66179c060616beb175c7935681ba339
Jul 17 18:06:15.071001 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:15.070913    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s" event={"ID":"5484266e-fb83-4cf4-99d2-46c331f78aba","Type":"ContainerStarted","Data":"823ed13fb76caa018ed7927e2acce48ab93520c822abbe53d9dc6d5d2e9f327a"}
Jul 17 18:06:15.071001 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:15.070971    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s" event={"ID":"5484266e-fb83-4cf4-99d2-46c331f78aba","Type":"ContainerStarted","Data":"b900d176cd1b5ab0a117884dbf763ac0e66179c060616beb175c7935681ba339"}
Jul 17 18:06:15.088570 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:15.088507    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s" podStartSLOduration=0.885431425 podStartE2EDuration="1.088487871s" totalImagesPullingTime="203.056446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:06:15.08720755 +0000 UTC m=+572.672683803" watchObservedRunningTime="2026-07-17 18:06:15.088487871 +0000 UTC m=+572.673964125"
Jul 17 18:06:15.345035 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:15.344932    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:06:18.006205 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:18.006158    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:18.006689 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:18.006227    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:18.018829 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:18.018802    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:18.099150 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:18.099122    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zth9g"
Jul 17 18:06:24.406334 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:24.406293    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:24.406779 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:24.406358    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:24.419009 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:24.418977    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:25.144933 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:25.144905    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-46n6s"
Jul 17 18:06:30.397117 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:30.397082    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:06:42.891939 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:42.891904    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:06:49.594893 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.594859    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"]
Jul 17 18:06:49.598478 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.598460    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.600950 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.600927    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 17 18:06:49.608610 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.608580    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"]
Jul 17 18:06:49.735356 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.735316    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.735575 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.735378    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.735575 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.735457    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdx2v\" (UniqueName: \"kubernetes.io/projected/b90fd575-54c4-4dd8-ac57-7d804b53a37d-kube-api-access-jdx2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.735575 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.735488    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b90fd575-54c4-4dd8-ac57-7d804b53a37d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.735575 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.735514    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.735836 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.735598    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837015 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.836965    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdx2v\" (UniqueName: \"kubernetes.io/projected/b90fd575-54c4-4dd8-ac57-7d804b53a37d-kube-api-access-jdx2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837029    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b90fd575-54c4-4dd8-ac57-7d804b53a37d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837056    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837096    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837128    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837176    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837581 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837549    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837757 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837710    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.837964 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.837942    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.839675 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.839624    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b90fd575-54c4-4dd8-ac57-7d804b53a37d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.839805 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.839787    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b90fd575-54c4-4dd8-ac57-7d804b53a37d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.844962 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.844902    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdx2v\" (UniqueName: \"kubernetes.io/projected/b90fd575-54c4-4dd8-ac57-7d804b53a37d-kube-api-access-jdx2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n\" (UID: \"b90fd575-54c4-4dd8-ac57-7d804b53a37d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:49.908193 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:49.908152    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:50.054236 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:50.054205    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"]
Jul 17 18:06:50.055734 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:06:50.055707    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb90fd575_54c4_4dd8_ac57_7d804b53a37d.slice/crio-aa76660357a135b49e7f9d2b12c9ace3990673b47b916d921d7cb7c820103b43 WatchSource:0}: Error finding container aa76660357a135b49e7f9d2b12c9ace3990673b47b916d921d7cb7c820103b43: Status 404 returned error can't find the container with id aa76660357a135b49e7f9d2b12c9ace3990673b47b916d921d7cb7c820103b43
Jul 17 18:06:50.057149 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:50.057131    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 18:06:50.280841 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:50.280807    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n" event={"ID":"b90fd575-54c4-4dd8-ac57-7d804b53a37d","Type":"ContainerStarted","Data":"aa76660357a135b49e7f9d2b12c9ace3990673b47b916d921d7cb7c820103b43"}
Jul 17 18:06:51.288316 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:51.288284    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n" event={"ID":"b90fd575-54c4-4dd8-ac57-7d804b53a37d","Type":"ContainerStarted","Data":"52e491af44fc40a4b7916789313562131b975a7c48007e27eee3e43b548c718e"}
Jul 17 18:06:51.307706 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:51.307634    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n" podStartSLOduration=2.032603704 podStartE2EDuration="2.307605825s" totalImagesPullingTime="275.002121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:06:51.306499002 +0000 UTC m=+608.891975253" watchObservedRunningTime="2026-07-17 18:06:51.307605825 +0000 UTC m=+608.893082076"
Jul 17 18:06:52.994710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:52.994664    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:06:59.909352 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:59.909305    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:59.909805 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:59.909369    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:06:59.922308 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:06:59.922278    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:07:00.351206 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:00.351171    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n"
Jul 17 18:07:45.088549 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.088509    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bdd7ff788-zsqkf"]
Jul 17 18:07:45.092126 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.092107    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.096128 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.096102    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zxtj5\""
Jul 17 18:07:45.096270 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.096151    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 17 18:07:45.102158 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.102132    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bdd7ff788-zsqkf"]
Jul 17 18:07:45.232121 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.232088    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9z5l\" (UniqueName: \"kubernetes.io/projected/da2eb1e6-14d0-4a3f-b9e3-ec647df07559-kube-api-access-q9z5l\") pod \"authorino-bdd7ff788-zsqkf\" (UID: \"da2eb1e6-14d0-4a3f-b9e3-ec647df07559\") " pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.232292 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.232231    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da2eb1e6-14d0-4a3f-b9e3-ec647df07559-tls-cert\") pod \"authorino-bdd7ff788-zsqkf\" (UID: \"da2eb1e6-14d0-4a3f-b9e3-ec647df07559\") " pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.333266 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.333224    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da2eb1e6-14d0-4a3f-b9e3-ec647df07559-tls-cert\") pod \"authorino-bdd7ff788-zsqkf\" (UID: \"da2eb1e6-14d0-4a3f-b9e3-ec647df07559\") " pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.333473 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.333312    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9z5l\" (UniqueName: \"kubernetes.io/projected/da2eb1e6-14d0-4a3f-b9e3-ec647df07559-kube-api-access-q9z5l\") pod \"authorino-bdd7ff788-zsqkf\" (UID: \"da2eb1e6-14d0-4a3f-b9e3-ec647df07559\") " pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.335686 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.335661    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da2eb1e6-14d0-4a3f-b9e3-ec647df07559-tls-cert\") pod \"authorino-bdd7ff788-zsqkf\" (UID: \"da2eb1e6-14d0-4a3f-b9e3-ec647df07559\") " pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.342652 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.342580    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9z5l\" (UniqueName: \"kubernetes.io/projected/da2eb1e6-14d0-4a3f-b9e3-ec647df07559-kube-api-access-q9z5l\") pod \"authorino-bdd7ff788-zsqkf\" (UID: \"da2eb1e6-14d0-4a3f-b9e3-ec647df07559\") " pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.403558 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.403517    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bdd7ff788-zsqkf"
Jul 17 18:07:45.573932 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.573902    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bdd7ff788-zsqkf"]
Jul 17 18:07:45.575837 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:07:45.575810    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda2eb1e6_14d0_4a3f_b9e3_ec647df07559.slice/crio-fddc9fe65738f41cb3b8fb65cbe7dad26ca0d561f938204a15be7e21bf7cfbe3 WatchSource:0}: Error finding container fddc9fe65738f41cb3b8fb65cbe7dad26ca0d561f938204a15be7e21bf7cfbe3: Status 404 returned error can't find the container with id fddc9fe65738f41cb3b8fb65cbe7dad26ca0d561f938204a15be7e21bf7cfbe3
Jul 17 18:07:45.599126 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:45.599039    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bdd7ff788-zsqkf" event={"ID":"da2eb1e6-14d0-4a3f-b9e3-ec647df07559","Type":"ContainerStarted","Data":"fddc9fe65738f41cb3b8fb65cbe7dad26ca0d561f938204a15be7e21bf7cfbe3"}
Jul 17 18:07:46.607579 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:46.607534    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bdd7ff788-zsqkf" event={"ID":"da2eb1e6-14d0-4a3f-b9e3-ec647df07559","Type":"ContainerStarted","Data":"a64fc2f8b6c74879353b3395be9e76199f3272b604657d6dc9f8cfbd3c833fcf"}
Jul 17 18:07:46.637657 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:07:46.637588    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-bdd7ff788-zsqkf" podStartSLOduration=1.095058958 podStartE2EDuration="1.637566317s" totalImagesPullingTime="542.507359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:07:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:07:46.636233653 +0000 UTC m=+664.221709906" watchObservedRunningTime="2026-07-17 18:07:46.637566317 +0000 UTC m=+664.223042571"
Jul 17 18:08:15.992280 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:08:15.992242    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:08:26.585873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:08:26.585835    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:08:37.203886 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:08:37.203790    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:10:15.408597 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:10:15.408494    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:10:26.892379 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:10:26.892347    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:11:04.896579 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:11:04.896545    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:11:17.089083 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:11:17.089046    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:11:26.094371 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:11:26.094327    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:11:33.394089 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:11:33.394055    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:11:42.891117 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:11:42.891071    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:11:52.299932 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:11:52.299889    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:12:04.492273 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:12:04.492241    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:12:14.388382 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:12:14.388344    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:12:36.686212 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:12:36.686176    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:12:46.097132 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:12:46.097095    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:13:08.697960 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:13:08.697870    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:13:19.296332 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:13:19.296299    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:13:36.107367 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:13:36.107327    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:13:45.401084 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:13:45.401042    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:13:55.293316 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:13:55.293281    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:14:04.916175 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:04.916136    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:14:15.487931 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:15.487894    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:14:40.454477 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:40.454385    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"]
Jul 17 18:14:40.455013 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:40.454610    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv" podUID="055b1caa-1462-4d2f-8848-c4e65728f6f5" containerName="manager" containerID="cri-o://08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e" gracePeriod=10
Jul 17 18:14:43.511195 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:43.511169    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:14:43.614741 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:43.614654    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfdxn\" (UniqueName: \"kubernetes.io/projected/055b1caa-1462-4d2f-8848-c4e65728f6f5-kube-api-access-nfdxn\") pod \"055b1caa-1462-4d2f-8848-c4e65728f6f5\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") "
Jul 17 18:14:43.614741 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:43.614728    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/055b1caa-1462-4d2f-8848-c4e65728f6f5-extensions-socket-volume\") pod \"055b1caa-1462-4d2f-8848-c4e65728f6f5\" (UID: \"055b1caa-1462-4d2f-8848-c4e65728f6f5\") "
Jul 17 18:14:43.615100 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:43.615080    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/055b1caa-1462-4d2f-8848-c4e65728f6f5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "055b1caa-1462-4d2f-8848-c4e65728f6f5" (UID: "055b1caa-1462-4d2f-8848-c4e65728f6f5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:14:43.616677 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:43.616658    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/055b1caa-1462-4d2f-8848-c4e65728f6f5-kube-api-access-nfdxn" (OuterVolumeSpecName: "kube-api-access-nfdxn") pod "055b1caa-1462-4d2f-8848-c4e65728f6f5" (UID: "055b1caa-1462-4d2f-8848-c4e65728f6f5"). InnerVolumeSpecName "kube-api-access-nfdxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:14:43.716039 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:43.716007    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/055b1caa-1462-4d2f-8848-c4e65728f6f5-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:14:43.716039 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:43.716034    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfdxn\" (UniqueName: \"kubernetes.io/projected/055b1caa-1462-4d2f-8848-c4e65728f6f5-kube-api-access-nfdxn\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:14:44.060340 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.060311    2555 generic.go:356] "Generic (PLEG): container finished" podID="055b1caa-1462-4d2f-8848-c4e65728f6f5" containerID="08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e" exitCode=0
Jul 17 18:14:44.060505 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.060354    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv" event={"ID":"055b1caa-1462-4d2f-8848-c4e65728f6f5","Type":"ContainerDied","Data":"08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e"}
Jul 17 18:14:44.060505 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.060391    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv" event={"ID":"055b1caa-1462-4d2f-8848-c4e65728f6f5","Type":"ContainerDied","Data":"d88cb9799f208efe86d3b01a9847a3af3cf43bea161a5774e1dccb13afe9ad20"}
Jul 17 18:14:44.060505 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.060413    2555 scope.go:117] "RemoveContainer" containerID="08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e"
Jul 17 18:14:44.060505 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.060355    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"
Jul 17 18:14:44.077669 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.077624    2555 scope.go:117] "RemoveContainer" containerID="08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e"
Jul 17 18:14:44.077967 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:14:44.077945    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e\": container with ID starting with 08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e not found: ID does not exist" containerID="08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e"
Jul 17 18:14:44.078052 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.077976    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e"} err="failed to get container status \"08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e\": rpc error: code = NotFound desc = could not find container \"08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e\": container with ID starting with 08a16a7f9c95bdcbc0d7ca8ffee4c1cec950de175789de1ad772e6c36055217e not found: ID does not exist"
Jul 17 18:14:44.086308 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.086283    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"]
Jul 17 18:14:44.089774 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.089752    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rrtv"]
Jul 17 18:14:44.944253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:14:44.944218    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="055b1caa-1462-4d2f-8848-c4e65728f6f5" path="/var/lib/kubelet/pods/055b1caa-1462-4d2f-8848-c4e65728f6f5/volumes"
Jul 17 18:15:00.152801 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.152767    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv"]
Jul 17 18:15:00.153445 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.153428    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="055b1caa-1462-4d2f-8848-c4e65728f6f5" containerName="manager"
Jul 17 18:15:00.153522 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.153448    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="055b1caa-1462-4d2f-8848-c4e65728f6f5" containerName="manager"
Jul 17 18:15:00.153565 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.153549    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="055b1caa-1462-4d2f-8848-c4e65728f6f5" containerName="manager"
Jul 17 18:15:00.158395 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.158366    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv"
Jul 17 18:15:00.160998 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.160970    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m6gj\""
Jul 17 18:15:00.161315 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.161289    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv"]
Jul 17 18:15:00.170085 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.170061    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv"
Jul 17 18:15:00.308585 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.308552    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv"]
Jul 17 18:15:00.310231 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:15:00.310202    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248a1faa_0782_45cc_8401_c751cfc42780.slice/crio-2fc8cd2d2d622d342e694ab1c5f05808162827d197cf28a5d7b6986dad1139ea WatchSource:0}: Error finding container 2fc8cd2d2d622d342e694ab1c5f05808162827d197cf28a5d7b6986dad1139ea: Status 404 returned error can't find the container with id 2fc8cd2d2d622d342e694ab1c5f05808162827d197cf28a5d7b6986dad1139ea
Jul 17 18:15:00.312191 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:00.312176    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 18:15:01.162355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:01.162322    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv" event={"ID":"248a1faa-0782-45cc-8401-c751cfc42780","Type":"ContainerStarted","Data":"2fc8cd2d2d622d342e694ab1c5f05808162827d197cf28a5d7b6986dad1139ea"}
Jul 17 18:15:03.175956 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:03.175908    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv" event={"ID":"248a1faa-0782-45cc-8401-c751cfc42780","Type":"ContainerStarted","Data":"0d0c600eeddd4182d201ec2d9ad9fa612b6b0f316e9415409989d18087ad3450"}
Jul 17 18:15:03.192978 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:03.192860    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv" podStartSLOduration=0.848578446 podStartE2EDuration="3.192839541s" totalImagesPullingTime="2.344261095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:03.190757822 +0000 UTC m=+1100.776234075" watchObservedRunningTime="2026-07-17 18:15:03.192839541 +0000 UTC m=+1100.778315793"
Jul 17 18:15:04.183171 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:04.183135    2555 generic.go:356] "Generic (PLEG): container finished" podID="248a1faa-0782-45cc-8401-c751cfc42780" containerID="0d0c600eeddd4182d201ec2d9ad9fa612b6b0f316e9415409989d18087ad3450" exitCode=0
Jul 17 18:15:04.183676 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:04.183228    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv" event={"ID":"248a1faa-0782-45cc-8401-c751cfc42780","Type":"ContainerDied","Data":"0d0c600eeddd4182d201ec2d9ad9fa612b6b0f316e9415409989d18087ad3450"}
Jul 17 18:15:05.329629 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:05.329601    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv"
Jul 17 18:15:06.196302 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:06.196276    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv"
Jul 17 18:15:06.196302 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:06.196302    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738535-25qfv" event={"ID":"248a1faa-0782-45cc-8401-c751cfc42780","Type":"ContainerDied","Data":"2fc8cd2d2d622d342e694ab1c5f05808162827d197cf28a5d7b6986dad1139ea"}
Jul 17 18:15:06.196520 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:06.196342    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fc8cd2d2d622d342e694ab1c5f05808162827d197cf28a5d7b6986dad1139ea"
Jul 17 18:15:41.583954 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.583907    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"]
Jul 17 18:15:41.584478 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.584465    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="248a1faa-0782-45cc-8401-c751cfc42780" containerName="cleanup"
Jul 17 18:15:41.584524 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.584481    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="248a1faa-0782-45cc-8401-c751cfc42780" containerName="cleanup"
Jul 17 18:15:41.584573 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.584559    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="248a1faa-0782-45cc-8401-c751cfc42780" containerName="cleanup"
Jul 17 18:15:41.586539 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.586521    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:41.589473 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.589454    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nbkwq\""
Jul 17 18:15:41.601258 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.601219    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"]
Jul 17 18:15:41.635737 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.635702    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f02c96ff-7c24-42c0-8937-49d85a643ff7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g4dpw\" (UID: \"f02c96ff-7c24-42c0-8937-49d85a643ff7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:41.635935 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.635820    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kqq2\" (UniqueName: \"kubernetes.io/projected/f02c96ff-7c24-42c0-8937-49d85a643ff7-kube-api-access-4kqq2\") pod \"kuadrant-operator-controller-manager-86878c769f-g4dpw\" (UID: \"f02c96ff-7c24-42c0-8937-49d85a643ff7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:41.736479 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.736437    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kqq2\" (UniqueName: \"kubernetes.io/projected/f02c96ff-7c24-42c0-8937-49d85a643ff7-kube-api-access-4kqq2\") pod \"kuadrant-operator-controller-manager-86878c769f-g4dpw\" (UID: \"f02c96ff-7c24-42c0-8937-49d85a643ff7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:41.736709 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.736585    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f02c96ff-7c24-42c0-8937-49d85a643ff7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g4dpw\" (UID: \"f02c96ff-7c24-42c0-8937-49d85a643ff7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:41.736966 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.736948    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f02c96ff-7c24-42c0-8937-49d85a643ff7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g4dpw\" (UID: \"f02c96ff-7c24-42c0-8937-49d85a643ff7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:41.747060 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.747031    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kqq2\" (UniqueName: \"kubernetes.io/projected/f02c96ff-7c24-42c0-8937-49d85a643ff7-kube-api-access-4kqq2\") pod \"kuadrant-operator-controller-manager-86878c769f-g4dpw\" (UID: \"f02c96ff-7c24-42c0-8937-49d85a643ff7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:41.897173 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:41.897076    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:42.045992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:42.045968    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"]
Jul 17 18:15:42.047907 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:15:42.047884    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02c96ff_7c24_42c0_8937_49d85a643ff7.slice/crio-8b448359a70242a807a1bfccd96ea8f0cc2107961f50933761b02fb5cdeec8ae WatchSource:0}: Error finding container 8b448359a70242a807a1bfccd96ea8f0cc2107961f50933761b02fb5cdeec8ae: Status 404 returned error can't find the container with id 8b448359a70242a807a1bfccd96ea8f0cc2107961f50933761b02fb5cdeec8ae
Jul 17 18:15:42.415374 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:42.415334    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw" event={"ID":"f02c96ff-7c24-42c0-8937-49d85a643ff7","Type":"ContainerStarted","Data":"173f57964c46a0cf3d46602c638a2d567fc16d27ce3c4a484c5aafc1bcb2fb07"}
Jul 17 18:15:42.415564 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:42.415390    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw" event={"ID":"f02c96ff-7c24-42c0-8937-49d85a643ff7","Type":"ContainerStarted","Data":"8b448359a70242a807a1bfccd96ea8f0cc2107961f50933761b02fb5cdeec8ae"}
Jul 17 18:15:42.415564 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:42.415464    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:15:42.438885 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:42.438824    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw" podStartSLOduration=1.4388021100000001 podStartE2EDuration="1.43880211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:15:42.436440886 +0000 UTC m=+1140.021917140" watchObservedRunningTime="2026-07-17 18:15:42.43880211 +0000 UTC m=+1140.024278363"
Jul 17 18:15:53.424388 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:15:53.424359    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g4dpw"
Jul 17 18:16:03.390124 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:16:03.390085    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:16:15.498836 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:16:15.498740    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:16:24.494474 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:16:24.494438    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:16:37.794621 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:16:37.794583    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:16:47.586910 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:16:47.586876    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:16:56.887817 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:16:56.887785    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:17:07.289815 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:17:07.289777    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:17:17.198241 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:17:17.198202    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:17:28.792428 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:17:28.792390    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:17:37.694269 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:17:37.694184    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:17:52.391334 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:17:52.391291    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:18:09.714222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:18:09.714179    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:18:19.096663 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:18:19.096611    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:18:29.695118 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:18:29.695075    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:18:41.993449 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:18:41.993408    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:18:51.793879 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:18:51.793832    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:19:04.694670 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:19:04.694561    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:19:14.396010 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:19:14.395964    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:19:24.197769 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:19:24.197724    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:19:34.090655 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:19:34.090598    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:19:43.807573 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:19:43.807533    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:19:53.987515 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:19:53.987478    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:20:04.888375 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:20:04.888331    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:20:16.993178 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:20:16.993128    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:20:32.393370 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:20:32.393337    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:20:43.595272 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:20:43.595185    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:20:55.789238 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:20:55.789199    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:21:12.391267 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:21:12.391229    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:21:27.297653 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:21:27.297604    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:21:36.891492 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:21:36.891450    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:21:49.712002 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:21:49.711959    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:22:04.004189 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:22:04.004155    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:22:15.096906 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:22:15.096869    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:25:19.589580 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:25:19.589491    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:25:32.892304 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:25:32.892267    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:26:37.491169 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:37.491083    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:26:51.500665 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:51.500605    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:26:56.688940 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.688899    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"]
Jul 17 18:26:56.692756 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.692738    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"
Jul 17 18:26:56.695057 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.695017    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m6gj\""
Jul 17 18:26:56.695232 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.695208    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 18:26:56.699548 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.699520    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"]
Jul 17 18:26:56.729251 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.729207    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdf9421-712e-436c-b0b2-47a7dd3b9824-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-02bfc37f-v2642\" (UID: \"dfdf9421-712e-436c-b0b2-47a7dd3b9824\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"
Jul 17 18:26:56.829982 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.829939    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdf9421-712e-436c-b0b2-47a7dd3b9824-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-02bfc37f-v2642\" (UID: \"dfdf9421-712e-436c-b0b2-47a7dd3b9824\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"
Jul 17 18:26:56.830616 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:56.830595    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdf9421-712e-436c-b0b2-47a7dd3b9824-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-02bfc37f-v2642\" (UID: \"dfdf9421-712e-436c-b0b2-47a7dd3b9824\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"
Jul 17 18:26:57.005685 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:57.005614    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"
Jul 17 18:26:57.149054 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:57.149029    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"]
Jul 17 18:26:57.151262 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:26:57.151235    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfdf9421_712e_436c_b0b2_47a7dd3b9824.slice/crio-36f37dc5d8bfb743426e73d1a34587e5d8700b57230d4037df0d1a996f406a45 WatchSource:0}: Error finding container 36f37dc5d8bfb743426e73d1a34587e5d8700b57230d4037df0d1a996f406a45: Status 404 returned error can't find the container with id 36f37dc5d8bfb743426e73d1a34587e5d8700b57230d4037df0d1a996f406a45
Jul 17 18:26:57.463904 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:57.463872    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" event={"ID":"dfdf9421-712e-436c-b0b2-47a7dd3b9824","Type":"ContainerStarted","Data":"36f37dc5d8bfb743426e73d1a34587e5d8700b57230d4037df0d1a996f406a45"}
Jul 17 18:26:58.471503 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:58.471410    2555 generic.go:356] "Generic (PLEG): container finished" podID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerID="95dfdbe1d39a0fa033c449b21de9340d476ebfe9ae75425b3d657d37b5c7d421" exitCode=6
Jul 17 18:26:58.471896 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:58.471511    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" event={"ID":"dfdf9421-712e-436c-b0b2-47a7dd3b9824","Type":"ContainerDied","Data":"95dfdbe1d39a0fa033c449b21de9340d476ebfe9ae75425b3d657d37b5c7d421"}
Jul 17 18:26:58.471896 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:58.471709    2555 scope.go:117] "RemoveContainer" containerID="95dfdbe1d39a0fa033c449b21de9340d476ebfe9ae75425b3d657d37b5c7d421"
Jul 17 18:26:59.479803 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:59.479720    2555 generic.go:356] "Generic (PLEG): container finished" podID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerID="1af0be2e77c8e5f93d5798c611b9fba8375620f98cb73ea58b6704ed9542a7e8" exitCode=6
Jul 17 18:26:59.480238 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:59.479809    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" event={"ID":"dfdf9421-712e-436c-b0b2-47a7dd3b9824","Type":"ContainerDied","Data":"1af0be2e77c8e5f93d5798c611b9fba8375620f98cb73ea58b6704ed9542a7e8"}
Jul 17 18:26:59.480238 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:59.479882    2555 scope.go:117] "RemoveContainer" containerID="95dfdbe1d39a0fa033c449b21de9340d476ebfe9ae75425b3d657d37b5c7d421"
Jul 17 18:26:59.480238 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:26:59.480055    2555 scope.go:117] "RemoveContainer" containerID="1af0be2e77c8e5f93d5798c611b9fba8375620f98cb73ea58b6704ed9542a7e8"
Jul 17 18:26:59.480409 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:26:59.480295    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-02bfc37f-v2642_odh-ai-gateway-infra(dfdf9421-712e-436c-b0b2-47a7dd3b9824)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824"
Jul 17 18:27:00.488375 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:00.488348    2555 scope.go:117] "RemoveContainer" containerID="1af0be2e77c8e5f93d5798c611b9fba8375620f98cb73ea58b6704ed9542a7e8"
Jul 17 18:27:00.488891 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:27:00.488591    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-02bfc37f-v2642_odh-ai-gateway-infra(dfdf9421-712e-436c-b0b2-47a7dd3b9824)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824"
Jul 17 18:27:15.938098 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:15.938053    2555 scope.go:117] "RemoveContainer" containerID="1af0be2e77c8e5f93d5798c611b9fba8375620f98cb73ea58b6704ed9542a7e8"
Jul 17 18:27:16.598329 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:16.598299    2555 generic.go:356] "Generic (PLEG): container finished" podID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerID="8be01bbc2c8200ff462150da5fd141d15227dc8069e866e9bca1d23d90f347cf" exitCode=6
Jul 17 18:27:16.598505 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:16.598372    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" event={"ID":"dfdf9421-712e-436c-b0b2-47a7dd3b9824","Type":"ContainerDied","Data":"8be01bbc2c8200ff462150da5fd141d15227dc8069e866e9bca1d23d90f347cf"}
Jul 17 18:27:16.598505 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:16.598438    2555 scope.go:117] "RemoveContainer" containerID="1af0be2e77c8e5f93d5798c611b9fba8375620f98cb73ea58b6704ed9542a7e8"
Jul 17 18:27:16.598769 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:16.598753    2555 scope.go:117] "RemoveContainer" containerID="8be01bbc2c8200ff462150da5fd141d15227dc8069e866e9bca1d23d90f347cf"
Jul 17 18:27:16.599033 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:27:16.599006    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-02bfc37f-v2642_odh-ai-gateway-infra(dfdf9421-712e-436c-b0b2-47a7dd3b9824)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824"
Jul 17 18:27:17.643162 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:17.643122    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"]
Jul 17 18:27:17.788612 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:17.788583    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"
Jul 17 18:27:17.937222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:17.937119    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdf9421-712e-436c-b0b2-47a7dd3b9824-maas-api-service-ca\") pod \"dfdf9421-712e-436c-b0b2-47a7dd3b9824\" (UID: \"dfdf9421-712e-436c-b0b2-47a7dd3b9824\") "
Jul 17 18:27:17.937487 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:17.937466    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfdf9421-712e-436c-b0b2-47a7dd3b9824-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dfdf9421-712e-436c-b0b2-47a7dd3b9824" (UID: "dfdf9421-712e-436c-b0b2-47a7dd3b9824"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 18:27:18.038291 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:18.038259    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdf9421-712e-436c-b0b2-47a7dd3b9824-maas-api-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:27:18.612939 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:18.612903    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642" event={"ID":"dfdf9421-712e-436c-b0b2-47a7dd3b9824","Type":"ContainerDied","Data":"36f37dc5d8bfb743426e73d1a34587e5d8700b57230d4037df0d1a996f406a45"}
Jul 17 18:27:18.612939 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:18.612932    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"
Jul 17 18:27:18.613218 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:18.612964    2555 scope.go:117] "RemoveContainer" containerID="8be01bbc2c8200ff462150da5fd141d15227dc8069e866e9bca1d23d90f347cf"
Jul 17 18:27:18.635035 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:18.635006    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"]
Jul 17 18:27:18.639286 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:18.639259    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-02bfc37f-v2642"]
Jul 17 18:27:18.943804 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:27:18.943723    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" path="/var/lib/kubelet/pods/dfdf9421-712e-436c-b0b2-47a7dd3b9824/volumes"
Jul 17 18:32:55.889055 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:32:55.888963    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:33:08.891702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:33:08.891656    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:36:13.396657 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:36:13.396608    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:37:45.391516 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:37:45.391475    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:37:54.490486 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:37:54.490449    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:38:07.926943 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:38:07.926900    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:38:27.483543 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:38:27.483505    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:39:09.193797 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:09.193763    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:39:23.300365 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300331    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"]
Jul 17 18:39:23.300884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300791    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.300884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300804    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.300884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300821    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.300884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300827    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.300884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300844    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.300884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300850    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.301081 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300912    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.301081 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300919    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.301081 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.300926    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfdf9421-712e-436c-b0b2-47a7dd3b9824" containerName="revoke-keys"
Jul 17 18:39:23.304030 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.304013    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.305691 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.305659    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-dshm\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.305849 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.305739    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh5t9\" (UniqueName: \"kubernetes.io/projected/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-kube-api-access-hh5t9\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.305849 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.305781    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-model-cache\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.305849 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.305838    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-home\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.305994 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.305861    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tmp-dir\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.305994 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.305928    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tls-certs\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.307791 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.307565    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-186d6157\"/\"kube-root-ca.crt\""
Jul 17 18:39:23.307791 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.307612    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-186d6157\"/\"openshift-service-ca.crt\""
Jul 17 18:39:23.307791 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.307671    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-186d6157\"/\"default-dockercfg-qmvcq\""
Jul 17 18:39:23.308012 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.307849    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-186d6157\"/\"e2e-lifecycle-model-186d6157-kserve-self-signed-certs\""
Jul 17 18:39:23.312164 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.312028    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"]
Jul 17 18:39:23.406991 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.406955    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tls-certs\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407014    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-dshm\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407066    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh5t9\" (UniqueName: \"kubernetes.io/projected/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-kube-api-access-hh5t9\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407087    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-model-cache\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407138    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-home\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407345 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407160    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tmp-dir\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407583 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407556    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-model-cache\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407567    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tmp-dir\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.407710 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.407673    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-home\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.409334 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.409310    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-dshm\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.409891 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.409874    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tls-certs\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.415083 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.415048    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh5t9\" (UniqueName: \"kubernetes.io/projected/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-kube-api-access-hh5t9\") pod \"e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") " pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.614898 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.614799    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:23.770506 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.770480    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"]
Jul 17 18:39:23.771977 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:39:23.771938    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode26fa466_8f4e_4e18_a877_5c28b7d8fc7c.slice/crio-d4a1200061d1c816513624d8880b653f6b6564e97475cb881ca2886f6c4d28a8 WatchSource:0}: Error finding container d4a1200061d1c816513624d8880b653f6b6564e97475cb881ca2886f6c4d28a8: Status 404 returned error can't find the container with id d4a1200061d1c816513624d8880b653f6b6564e97475cb881ca2886f6c4d28a8
Jul 17 18:39:23.901269 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.901162    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2" event={"ID":"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c","Type":"ContainerStarted","Data":"ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134"}
Jul 17 18:39:23.901269 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.901211    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2" event={"ID":"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c","Type":"ContainerStarted","Data":"d4a1200061d1c816513624d8880b653f6b6564e97475cb881ca2886f6c4d28a8"}
Jul 17 18:39:23.918299 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:23.918239    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2" podStartSLOduration=0.91821624 podStartE2EDuration="918.21624ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:39:23.91769595 +0000 UTC m=+2561.503172201" watchObservedRunningTime="2026-07-17 18:39:23.91821624 +0000 UTC m=+2561.503692497"
Jul 17 18:39:26.787734 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:26.787696    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:39:33.615771 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:33.615733    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:33.616283 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:33.616023    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:33.628744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:33.628712    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:33.971147 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:33.971063    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:39:42.103899 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:42.103857    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:39:59.288489 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:39:59.288449    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:40:05.454142 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:05.454056    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"]
Jul 17 18:40:20.475551 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.475441    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2" podUID="e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" containerName="main" containerID="cri-o://ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134" gracePeriod=45
Jul 17 18:40:20.740872 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.740802    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:40:20.845714 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.845668    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tmp-dir\") pod \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") "
Jul 17 18:40:20.845714 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.845725    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh5t9\" (UniqueName: \"kubernetes.io/projected/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-kube-api-access-hh5t9\") pod \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") "
Jul 17 18:40:20.845979 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.845765    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-model-cache\") pod \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") "
Jul 17 18:40:20.845979 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.845781    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tls-certs\") pod \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") "
Jul 17 18:40:20.845979 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.845820    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-dshm\") pod \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") "
Jul 17 18:40:20.845979 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.845836    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-home\") pod \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\" (UID: \"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c\") "
Jul 17 18:40:20.846179 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.846089    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" (UID: "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:40:20.846179 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.846097    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-model-cache" (OuterVolumeSpecName: "model-cache") pod "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" (UID: "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:40:20.846250 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.846169    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-home" (OuterVolumeSpecName: "home") pod "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" (UID: "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:40:20.848147 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.848114    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-dshm" (OuterVolumeSpecName: "dshm") pod "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" (UID: "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:40:20.848292 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.848196    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" (UID: "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:40:20.848292 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.848205    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-kube-api-access-hh5t9" (OuterVolumeSpecName: "kube-api-access-hh5t9") pod "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" (UID: "e26fa466-8f4e-4e18-a877-5c28b7d8fc7c"). InnerVolumeSpecName "kube-api-access-hh5t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:40:20.946918 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.946886    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:40:20.946918 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.946910    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:40:20.947187 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.946939    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:40:20.947187 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.946949    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh5t9\" (UniqueName: \"kubernetes.io/projected/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-kube-api-access-hh5t9\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:40:20.947187 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.946959    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:40:20.947187 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:20.946967    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:40:21.245209 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.245176    2555 generic.go:356] "Generic (PLEG): container finished" podID="e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" containerID="ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134" exitCode=0
Jul 17 18:40:21.245416 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.245228    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"
Jul 17 18:40:21.245416 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.245267    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2" event={"ID":"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c","Type":"ContainerDied","Data":"ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134"}
Jul 17 18:40:21.245416 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.245321    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2" event={"ID":"e26fa466-8f4e-4e18-a877-5c28b7d8fc7c","Type":"ContainerDied","Data":"d4a1200061d1c816513624d8880b653f6b6564e97475cb881ca2886f6c4d28a8"}
Jul 17 18:40:21.245416 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.245342    2555 scope.go:117] "RemoveContainer" containerID="ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134"
Jul 17 18:40:21.255083 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.255062    2555 scope.go:117] "RemoveContainer" containerID="ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134"
Jul 17 18:40:21.255436 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:40:21.255409    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134\": container with ID starting with ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134 not found: ID does not exist" containerID="ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134"
Jul 17 18:40:21.255538 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.255442    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134"} err="failed to get container status \"ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134\": rpc error: code = NotFound desc = could not find container \"ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134\": container with ID starting with ad0ce567d8e61d090953a6ee187221f4d558d21534cea4a6cf67c6c5e77ac134 not found: ID does not exist"
Jul 17 18:40:21.262744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.262716    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"]
Jul 17 18:40:21.267727 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:21.267694    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-186d6157/e2e-lifecycle-model-186d6157-kserve-599895dd8c-2ngj2"]
Jul 17 18:40:22.944545 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:22.944504    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" path="/var/lib/kubelet/pods/e26fa466-8f4e-4e18-a877-5c28b7d8fc7c/volumes"
Jul 17 18:40:58.796359 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:40:58.796311    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:41:09.899819 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:41:09.899776    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:41:58.806353 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:41:58.806305    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:42:10.388424 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.388383    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"]
Jul 17 18:42:10.388856 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.388836    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" containerName="main"
Jul 17 18:42:10.388856 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.388848    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" containerName="main"
Jul 17 18:42:10.388945 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.388934    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e26fa466-8f4e-4e18-a877-5c28b7d8fc7c" containerName="main"
Jul 17 18:42:10.392229 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.392209    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.395886 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.395856    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ddeacf\"/\"openshift-service-ca.crt\""
Jul 17 18:42:10.395886 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.395875    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ddeacf\"/\"kube-root-ca.crt\""
Jul 17 18:42:10.396094 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.395855    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ddeacf\"/\"test-model-ddeacf-kserve-self-signed-certs\""
Jul 17 18:42:10.396094 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.395855    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ddeacf\"/\"default-dockercfg-2z2zj\""
Jul 17 18:42:10.400586 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.400562    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"]
Jul 17 18:42:10.481922 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.481888    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-model-cache\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.482111 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.481988    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km2w4\" (UniqueName: \"kubernetes.io/projected/88e5748d-324f-4cdc-abce-3ddf9e12395d-kube-api-access-km2w4\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.482111 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.482030    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-tmp-dir\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.482111 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.482056    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-dshm\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.482111 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.482072    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88e5748d-324f-4cdc-abce-3ddf9e12395d-tls-certs\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.482111 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.482111    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-home\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.583493 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583444    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km2w4\" (UniqueName: \"kubernetes.io/projected/88e5748d-324f-4cdc-abce-3ddf9e12395d-kube-api-access-km2w4\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.583714 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583504    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-tmp-dir\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.583714 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583525    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-dshm\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.583714 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583539    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88e5748d-324f-4cdc-abce-3ddf9e12395d-tls-certs\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.583714 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583574    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-home\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.583714 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583602    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-model-cache\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.583972 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583944    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-tmp-dir\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.584037 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.583994    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-model-cache\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.584037 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.584027    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-home\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.586049 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.586019    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-dshm\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.586229 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.586210    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88e5748d-324f-4cdc-abce-3ddf9e12395d-tls-certs\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.592155 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.592123    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km2w4\" (UniqueName: \"kubernetes.io/projected/88e5748d-324f-4cdc-abce-3ddf9e12395d-kube-api-access-km2w4\") pod \"test-model-ddeacf-kserve-6b5cc7d58c-whq57\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") " pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.704411 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.704312    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:10.783440 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.783404    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:42:10.864302 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.864269    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"]
Jul 17 18:42:10.866062 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:42:10.866024    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88e5748d_324f_4cdc_abce_3ddf9e12395d.slice/crio-5d1739a111d047306dee2517f9e792c7b0922b82fc059c503d34ce8fa85728a3 WatchSource:0}: Error finding container 5d1739a111d047306dee2517f9e792c7b0922b82fc059c503d34ce8fa85728a3: Status 404 returned error can't find the container with id 5d1739a111d047306dee2517f9e792c7b0922b82fc059c503d34ce8fa85728a3
Jul 17 18:42:10.920088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:10.920055    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57" event={"ID":"88e5748d-324f-4cdc-abce-3ddf9e12395d","Type":"ContainerStarted","Data":"5d1739a111d047306dee2517f9e792c7b0922b82fc059c503d34ce8fa85728a3"}
Jul 17 18:42:11.927590 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:11.927542    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57" event={"ID":"88e5748d-324f-4cdc-abce-3ddf9e12395d","Type":"ContainerStarted","Data":"da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56"}
Jul 17 18:42:11.950321 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:11.950260    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57" podStartSLOduration=1.950241488 podStartE2EDuration="1.950241488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:42:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:42:11.949748217 +0000 UTC m=+2729.535224469" watchObservedRunningTime="2026-07-17 18:42:11.950241488 +0000 UTC m=+2729.535717750"
Jul 17 18:42:20.705074 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:20.705023    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:20.705534 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:20.705106    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:20.721749 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:20.721715    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:20.995811 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:20.995785    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:42:26.985560 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:26.985520    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:42:59.986577 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:42:59.986533    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:43:17.344240 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:17.344132    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"]
Jul 17 18:43:18.707474 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:18.707434    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:43:32.093987 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.093945    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:43:32.367167 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.367075    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57" podUID="88e5748d-324f-4cdc-abce-3ddf9e12395d" containerName="main" containerID="cri-o://da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56" gracePeriod=45
Jul 17 18:43:32.642011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.641976    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:43:32.768431 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768383    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-dshm\") pod \"88e5748d-324f-4cdc-abce-3ddf9e12395d\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") "
Jul 17 18:43:32.768431 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768429    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-home\") pod \"88e5748d-324f-4cdc-abce-3ddf9e12395d\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") "
Jul 17 18:43:32.768744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768482    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km2w4\" (UniqueName: \"kubernetes.io/projected/88e5748d-324f-4cdc-abce-3ddf9e12395d-kube-api-access-km2w4\") pod \"88e5748d-324f-4cdc-abce-3ddf9e12395d\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") "
Jul 17 18:43:32.768744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768501    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88e5748d-324f-4cdc-abce-3ddf9e12395d-tls-certs\") pod \"88e5748d-324f-4cdc-abce-3ddf9e12395d\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") "
Jul 17 18:43:32.768744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768551    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-model-cache\") pod \"88e5748d-324f-4cdc-abce-3ddf9e12395d\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") "
Jul 17 18:43:32.768744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768668    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-tmp-dir\") pod \"88e5748d-324f-4cdc-abce-3ddf9e12395d\" (UID: \"88e5748d-324f-4cdc-abce-3ddf9e12395d\") "
Jul 17 18:43:32.768967 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768819    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-home" (OuterVolumeSpecName: "home") pod "88e5748d-324f-4cdc-abce-3ddf9e12395d" (UID: "88e5748d-324f-4cdc-abce-3ddf9e12395d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:43:32.768967 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768884    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-model-cache" (OuterVolumeSpecName: "model-cache") pod "88e5748d-324f-4cdc-abce-3ddf9e12395d" (UID: "88e5748d-324f-4cdc-abce-3ddf9e12395d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:43:32.769042 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.768958    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "88e5748d-324f-4cdc-abce-3ddf9e12395d" (UID: "88e5748d-324f-4cdc-abce-3ddf9e12395d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:43:32.769093 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.769071    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:43:32.769146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.769095    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:43:32.769146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.769108    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:43:32.770956 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.770920    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-dshm" (OuterVolumeSpecName: "dshm") pod "88e5748d-324f-4cdc-abce-3ddf9e12395d" (UID: "88e5748d-324f-4cdc-abce-3ddf9e12395d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:43:32.770956 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.770942    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e5748d-324f-4cdc-abce-3ddf9e12395d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88e5748d-324f-4cdc-abce-3ddf9e12395d" (UID: "88e5748d-324f-4cdc-abce-3ddf9e12395d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:43:32.771098 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.770971    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88e5748d-324f-4cdc-abce-3ddf9e12395d-kube-api-access-km2w4" (OuterVolumeSpecName: "kube-api-access-km2w4") pod "88e5748d-324f-4cdc-abce-3ddf9e12395d" (UID: "88e5748d-324f-4cdc-abce-3ddf9e12395d"). InnerVolumeSpecName "kube-api-access-km2w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:43:32.870456 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.870359    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88e5748d-324f-4cdc-abce-3ddf9e12395d-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:43:32.870456 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.870395    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km2w4\" (UniqueName: \"kubernetes.io/projected/88e5748d-324f-4cdc-abce-3ddf9e12395d-kube-api-access-km2w4\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:43:32.870456 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:32.870408    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88e5748d-324f-4cdc-abce-3ddf9e12395d-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:43:33.444954 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.444916    2555 generic.go:356] "Generic (PLEG): container finished" podID="88e5748d-324f-4cdc-abce-3ddf9e12395d" containerID="da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56" exitCode=0
Jul 17 18:43:33.445384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.444968    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"
Jul 17 18:43:33.445384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.445003    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57" event={"ID":"88e5748d-324f-4cdc-abce-3ddf9e12395d","Type":"ContainerDied","Data":"da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56"}
Jul 17 18:43:33.445384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.445050    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57" event={"ID":"88e5748d-324f-4cdc-abce-3ddf9e12395d","Type":"ContainerDied","Data":"5d1739a111d047306dee2517f9e792c7b0922b82fc059c503d34ce8fa85728a3"}
Jul 17 18:43:33.445384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.445073    2555 scope.go:117] "RemoveContainer" containerID="da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56"
Jul 17 18:43:33.454757 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.454737    2555 scope.go:117] "RemoveContainer" containerID="da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56"
Jul 17 18:43:33.455099 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:43:33.455076    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56\": container with ID starting with da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56 not found: ID does not exist" containerID="da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56"
Jul 17 18:43:33.455164 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.455109    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56"} err="failed to get container status \"da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56\": rpc error: code = NotFound desc = could not find container \"da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56\": container with ID starting with da07d04354a02c5fde659a346bbaaaa33d69068c5398be92d1d81c884ab6fa56 not found: ID does not exist"
Jul 17 18:43:33.462373 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.462329    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"]
Jul 17 18:43:33.466381 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:33.466344    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-ddeacf/test-model-ddeacf-kserve-6b5cc7d58c-whq57"]
Jul 17 18:43:34.944227 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:34.944187    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88e5748d-324f-4cdc-abce-3ddf9e12395d" path="/var/lib/kubelet/pods/88e5748d-324f-4cdc-abce-3ddf9e12395d/volumes"
Jul 17 18:43:58.390240 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:43:58.390199    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:44:19.503945 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:44:19.503903    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:44:30.791656 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:44:30.791609    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:45:00.140885 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.140846    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"]
Jul 17 18:45:00.141542 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.141521    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88e5748d-324f-4cdc-abce-3ddf9e12395d" containerName="main"
Jul 17 18:45:00.141634 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.141545    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e5748d-324f-4cdc-abce-3ddf9e12395d" containerName="main"
Jul 17 18:45:00.141742 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.141695    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="88e5748d-324f-4cdc-abce-3ddf9e12395d" containerName="main"
Jul 17 18:45:00.144848 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.144825    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"
Jul 17 18:45:00.147947 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.147922    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8m6gj\""
Jul 17 18:45:00.153902 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.153609    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"]
Jul 17 18:45:00.157878 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.157851    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"
Jul 17 18:45:00.301082 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.301055    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"]
Jul 17 18:45:00.302771 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:45:00.302736    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40017df2_c38e_4ef3_9a7d_0ee8c25c2220.slice/crio-183eed169a8be1e97f89beb27f378b050a7134811ad0bd9c3e6d2a7dc0d78cbe WatchSource:0}: Error finding container 183eed169a8be1e97f89beb27f378b050a7134811ad0bd9c3e6d2a7dc0d78cbe: Status 404 returned error can't find the container with id 183eed169a8be1e97f89beb27f378b050a7134811ad0bd9c3e6d2a7dc0d78cbe
Jul 17 18:45:00.972107 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.972008    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" event={"ID":"40017df2-c38e-4ef3-9a7d-0ee8c25c2220","Type":"ContainerStarted","Data":"6cfbe8facf33cfe06e82574e51890118f5653787f464ed086d1f7bbdaa3bf6c2"}
Jul 17 18:45:00.972107 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.972067    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" event={"ID":"40017df2-c38e-4ef3-9a7d-0ee8c25c2220","Type":"ContainerStarted","Data":"183eed169a8be1e97f89beb27f378b050a7134811ad0bd9c3e6d2a7dc0d78cbe"}
Jul 17 18:45:00.988684 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:00.988613    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" podStartSLOduration=0.988596269 podStartE2EDuration="988.596269ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:45:00.987178458 +0000 UTC m=+2898.572654712" watchObservedRunningTime="2026-07-17 18:45:00.988596269 +0000 UTC m=+2898.574072520"
Jul 17 18:45:01.979319 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:01.979277    2555 generic.go:356] "Generic (PLEG): container finished" podID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerID="6cfbe8facf33cfe06e82574e51890118f5653787f464ed086d1f7bbdaa3bf6c2" exitCode=7
Jul 17 18:45:01.979747 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:01.979329    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" event={"ID":"40017df2-c38e-4ef3-9a7d-0ee8c25c2220","Type":"ContainerDied","Data":"6cfbe8facf33cfe06e82574e51890118f5653787f464ed086d1f7bbdaa3bf6c2"}
Jul 17 18:45:01.979747 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:01.979528    2555 scope.go:117] "RemoveContainer" containerID="6cfbe8facf33cfe06e82574e51890118f5653787f464ed086d1f7bbdaa3bf6c2"
Jul 17 18:45:02.987076 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:02.986988    2555 generic.go:356] "Generic (PLEG): container finished" podID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerID="07c7ee1229385c3e262cfd351696b205eaf429ddf651c72d8ff60c15b2f26ce8" exitCode=7
Jul 17 18:45:02.987076 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:02.987043    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" event={"ID":"40017df2-c38e-4ef3-9a7d-0ee8c25c2220","Type":"ContainerDied","Data":"07c7ee1229385c3e262cfd351696b205eaf429ddf651c72d8ff60c15b2f26ce8"}
Jul 17 18:45:02.987528 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:02.987090    2555 scope.go:117] "RemoveContainer" containerID="6cfbe8facf33cfe06e82574e51890118f5653787f464ed086d1f7bbdaa3bf6c2"
Jul 17 18:45:02.987528 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:02.987247    2555 scope.go:117] "RemoveContainer" containerID="07c7ee1229385c3e262cfd351696b205eaf429ddf651c72d8ff60c15b2f26ce8"
Jul 17 18:45:02.987528 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:45:02.987520    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l_odh-ai-gateway-infra(40017df2-c38e-4ef3-9a7d-0ee8c25c2220)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220"
Jul 17 18:45:03.995207 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:03.995183    2555 scope.go:117] "RemoveContainer" containerID="07c7ee1229385c3e262cfd351696b205eaf429ddf651c72d8ff60c15b2f26ce8"
Jul 17 18:45:03.995606 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:45:03.995414    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l_odh-ai-gateway-infra(40017df2-c38e-4ef3-9a7d-0ee8c25c2220)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220"
Jul 17 18:45:16.938355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:16.938322    2555 scope.go:117] "RemoveContainer" containerID="07c7ee1229385c3e262cfd351696b205eaf429ddf651c72d8ff60c15b2f26ce8"
Jul 17 18:45:18.079029 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:18.078996    2555 generic.go:356] "Generic (PLEG): container finished" podID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerID="225d8655386b8723f3d28c2ba61672c21ad6927096f7f434811e4a9b2023e677" exitCode=7
Jul 17 18:45:18.079439 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:18.079072    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" event={"ID":"40017df2-c38e-4ef3-9a7d-0ee8c25c2220","Type":"ContainerDied","Data":"225d8655386b8723f3d28c2ba61672c21ad6927096f7f434811e4a9b2023e677"}
Jul 17 18:45:18.079439 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:18.079132    2555 scope.go:117] "RemoveContainer" containerID="07c7ee1229385c3e262cfd351696b205eaf429ddf651c72d8ff60c15b2f26ce8"
Jul 17 18:45:18.079567 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:18.079445    2555 scope.go:117] "RemoveContainer" containerID="225d8655386b8723f3d28c2ba61672c21ad6927096f7f434811e4a9b2023e677"
Jul 17 18:45:18.079756 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:45:18.079732    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l_odh-ai-gateway-infra(40017df2-c38e-4ef3-9a7d-0ee8c25c2220)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220"
Jul 17 18:45:19.112131 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:19.112088    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"]
Jul 17 18:45:19.253567 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:19.253539    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"
Jul 17 18:45:20.093825 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:20.093796    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"
Jul 17 18:45:20.094021 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:20.093793    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l" event={"ID":"40017df2-c38e-4ef3-9a7d-0ee8c25c2220","Type":"ContainerDied","Data":"183eed169a8be1e97f89beb27f378b050a7134811ad0bd9c3e6d2a7dc0d78cbe"}
Jul 17 18:45:20.094021 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:20.093931    2555 scope.go:117] "RemoveContainer" containerID="225d8655386b8723f3d28c2ba61672c21ad6927096f7f434811e4a9b2023e677"
Jul 17 18:45:20.115631 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:20.115591    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"]
Jul 17 18:45:20.123832 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:20.123801    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a03eb5-29738565-sk52l"]
Jul 17 18:45:20.943576 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:45:20.943539    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" path="/var/lib/kubelet/pods/40017df2-c38e-4ef3-9a7d-0ee8c25c2220/volumes"
Jul 17 18:50:33.991549 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:50:33.991508    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:51:00.085547 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:00.085509    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:51:21.281498 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:21.281457    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:51:32.593771 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:32.593733    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:51:39.586774 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.586738    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"]
Jul 17 18:51:39.587372 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.587350    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:51:39.587372 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.587373    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:51:39.587495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.587422    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:51:39.587495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.587432    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:51:39.587576 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.587523    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:51:39.587576 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.587535    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:51:39.590865 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.590846    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.594365 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.594341    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"openshift-service-ca.crt\""
Jul 17 18:51:39.594515 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.594393    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"default-dockercfg-zlqbb\""
Jul 17 18:51:39.594515 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.594347    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"auth-test-model-7d6f90-kserve-self-signed-certs\""
Jul 17 18:51:39.594515 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.594341    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"kube-root-ca.crt\""
Jul 17 18:51:39.599889 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.599864    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"]
Jul 17 18:51:39.727662 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.727601    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftds7\" (UniqueName: \"kubernetes.io/projected/6cbd8b74-46a9-42e8-a426-c73dd64f173d-kube-api-access-ftds7\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.727844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.727732    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-dshm\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.727844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.727786    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-model-cache\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.727844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.727808    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tls-certs\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.727951 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.727891    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tmp-dir\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.727951 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.727931    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-home\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.828736 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.828693    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tmp-dir\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.828736 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.828738    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-home\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.828974 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.828783    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftds7\" (UniqueName: \"kubernetes.io/projected/6cbd8b74-46a9-42e8-a426-c73dd64f173d-kube-api-access-ftds7\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.828974 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.828850    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-dshm\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.829070 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.828979    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-model-cache\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.829070 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.829026    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tls-certs\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.829176 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.829154    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tmp-dir\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.829239 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.829196    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-home\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.829424 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.829402    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-model-cache\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.831152 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.831127    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-dshm\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.831439 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.831420    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tls-certs\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.842702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.839243    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftds7\" (UniqueName: \"kubernetes.io/projected/6cbd8b74-46a9-42e8-a426-c73dd64f173d-kube-api-access-ftds7\") pod \"auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") " pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:39.902105 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:39.902058    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:40.044420 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:40.044393    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"]
Jul 17 18:51:40.046411 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:51:40.046377    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cbd8b74_46a9_42e8_a426_c73dd64f173d.slice/crio-df0b6d66304df0d8f9bbdfe9d6082335260a17b9bc8ffb7c924b90c772c9dc51 WatchSource:0}: Error finding container df0b6d66304df0d8f9bbdfe9d6082335260a17b9bc8ffb7c924b90c772c9dc51: Status 404 returned error can't find the container with id df0b6d66304df0d8f9bbdfe9d6082335260a17b9bc8ffb7c924b90c772c9dc51
Jul 17 18:51:40.349835 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:40.349793    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8" event={"ID":"6cbd8b74-46a9-42e8-a426-c73dd64f173d","Type":"ContainerStarted","Data":"72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c"}
Jul 17 18:51:40.350010 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:40.349841    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8" event={"ID":"6cbd8b74-46a9-42e8-a426-c73dd64f173d","Type":"ContainerStarted","Data":"df0b6d66304df0d8f9bbdfe9d6082335260a17b9bc8ffb7c924b90c772c9dc51"}
Jul 17 18:51:40.369394 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:40.369255    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8" podStartSLOduration=1.369232822 podStartE2EDuration="1.369232822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:51:40.366908781 +0000 UTC m=+3297.952385033" watchObservedRunningTime="2026-07-17 18:51:40.369232822 +0000 UTC m=+3297.954709074"
Jul 17 18:51:42.592002 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:42.591966    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:51:49.902802 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:49.902759    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:49.903211 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:49.902823    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:49.915540 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:49.915513    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:51:50.426932 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:51:50.426894    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:52:08.685152 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:52:08.685064    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:52:26.789556 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:52:26.789508    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:52:43.887867 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:52:43.887825    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:52:59.385166 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:52:59.385129    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:53:18.594277 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:53:18.594229    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:53:37.785693 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:53:37.785613    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:53:55.188114 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:53:55.188077    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:54:10.433292 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:10.433257    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"]
Jul 17 18:54:14.485838 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:14.485801    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:54:25.457409 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.457365    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8" podUID="6cbd8b74-46a9-42e8-a426-c73dd64f173d" containerName="main" containerID="cri-o://72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c" gracePeriod=45
Jul 17 18:54:25.727174 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.727148    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:54:25.812177 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812142    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-model-cache\") pod \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") "
Jul 17 18:54:25.812355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812243    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftds7\" (UniqueName: \"kubernetes.io/projected/6cbd8b74-46a9-42e8-a426-c73dd64f173d-kube-api-access-ftds7\") pod \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") "
Jul 17 18:54:25.812355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812263    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tmp-dir\") pod \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") "
Jul 17 18:54:25.812355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812285    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-home\") pod \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") "
Jul 17 18:54:25.812355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812326    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tls-certs\") pod \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") "
Jul 17 18:54:25.812355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812352    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-dshm\") pod \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\" (UID: \"6cbd8b74-46a9-42e8-a426-c73dd64f173d\") "
Jul 17 18:54:25.812601 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812485    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-model-cache" (OuterVolumeSpecName: "model-cache") pod "6cbd8b74-46a9-42e8-a426-c73dd64f173d" (UID: "6cbd8b74-46a9-42e8-a426-c73dd64f173d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:54:25.812601 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812590    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-home" (OuterVolumeSpecName: "home") pod "6cbd8b74-46a9-42e8-a426-c73dd64f173d" (UID: "6cbd8b74-46a9-42e8-a426-c73dd64f173d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:54:25.812757 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812606    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6cbd8b74-46a9-42e8-a426-c73dd64f173d" (UID: "6cbd8b74-46a9-42e8-a426-c73dd64f173d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:54:25.812808 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812758    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:54:25.812808 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812775    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:54:25.812808 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.812783    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:54:25.814602 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.814567    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6cbd8b74-46a9-42e8-a426-c73dd64f173d" (UID: "6cbd8b74-46a9-42e8-a426-c73dd64f173d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:54:25.814602 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.814566    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-dshm" (OuterVolumeSpecName: "dshm") pod "6cbd8b74-46a9-42e8-a426-c73dd64f173d" (UID: "6cbd8b74-46a9-42e8-a426-c73dd64f173d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:54:25.814774 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.814610    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbd8b74-46a9-42e8-a426-c73dd64f173d-kube-api-access-ftds7" (OuterVolumeSpecName: "kube-api-access-ftds7") pod "6cbd8b74-46a9-42e8-a426-c73dd64f173d" (UID: "6cbd8b74-46a9-42e8-a426-c73dd64f173d"). InnerVolumeSpecName "kube-api-access-ftds7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:54:25.913253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.913211    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cbd8b74-46a9-42e8-a426-c73dd64f173d-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:54:25.913253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.913245    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftds7\" (UniqueName: \"kubernetes.io/projected/6cbd8b74-46a9-42e8-a426-c73dd64f173d-kube-api-access-ftds7\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:54:25.913253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:25.913258    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cbd8b74-46a9-42e8-a426-c73dd64f173d-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:54:26.353499 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.353458    2555 generic.go:356] "Generic (PLEG): container finished" podID="6cbd8b74-46a9-42e8-a426-c73dd64f173d" containerID="72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c" exitCode=0
Jul 17 18:54:26.353718 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.353521    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"
Jul 17 18:54:26.353718 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.353528    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8" event={"ID":"6cbd8b74-46a9-42e8-a426-c73dd64f173d","Type":"ContainerDied","Data":"72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c"}
Jul 17 18:54:26.353718 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.353570    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8" event={"ID":"6cbd8b74-46a9-42e8-a426-c73dd64f173d","Type":"ContainerDied","Data":"df0b6d66304df0d8f9bbdfe9d6082335260a17b9bc8ffb7c924b90c772c9dc51"}
Jul 17 18:54:26.353718 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.353592    2555 scope.go:117] "RemoveContainer" containerID="72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c"
Jul 17 18:54:26.364216 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.364187    2555 scope.go:117] "RemoveContainer" containerID="72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c"
Jul 17 18:54:26.364563 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:54:26.364543    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c\": container with ID starting with 72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c not found: ID does not exist" containerID="72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c"
Jul 17 18:54:26.364632 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.364572    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c"} err="failed to get container status \"72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c\": rpc error: code = NotFound desc = could not find container \"72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c\": container with ID starting with 72f413ae2cb90f2c1d5ac1bf140f384c2e0cd1c2fd8a4bfe997190184fb0453c not found: ID does not exist"
Jul 17 18:54:26.376760 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.376729    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"]
Jul 17 18:54:26.380455 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.380424    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/auth-test-model-7d6f90-kserve-7bfbd6c47-9qmk8"]
Jul 17 18:54:26.947139 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:26.947102    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cbd8b74-46a9-42e8-a426-c73dd64f173d" path="/var/lib/kubelet/pods/6cbd8b74-46a9-42e8-a426-c73dd64f173d/volumes"
Jul 17 18:54:37.293322 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.293279    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"]
Jul 17 18:54:37.294002 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.293983    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:54:37.294058 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.294004    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:54:37.294058 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.294027    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbd8b74-46a9-42e8-a426-c73dd64f173d" containerName="main"
Jul 17 18:54:37.294058 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.294032    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbd8b74-46a9-42e8-a426-c73dd64f173d" containerName="main"
Jul 17 18:54:37.294156 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.294097    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cbd8b74-46a9-42e8-a426-c73dd64f173d" containerName="main"
Jul 17 18:54:37.294156 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.294106    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="40017df2-c38e-4ef3-9a7d-0ee8c25c2220" containerName="cleanup"
Jul 17 18:54:37.298864 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.298836    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.302630 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.302605    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"sub-test-model-7d6f90-kserve-self-signed-certs\""
Jul 17 18:54:37.302884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.302676    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"openshift-service-ca.crt\""
Jul 17 18:54:37.302973 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.302714    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"default-dockercfg-zlqbb\""
Jul 17 18:54:37.302973 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.302723    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"kube-root-ca.crt\""
Jul 17 18:54:37.307371 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.307264    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"]
Jul 17 18:54:37.422679 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.422610    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ffb72e-4672-4f8f-8904-37144c6897cd-tls-certs\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.422873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.422716    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-home\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.422873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.422748    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-model-cache\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.422873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.422811    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-tmp-dir\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.422873 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.422844    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stgkh\" (UniqueName: \"kubernetes.io/projected/d0ffb72e-4672-4f8f-8904-37144c6897cd-kube-api-access-stgkh\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.423019 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.422917    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-dshm\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.524489 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.524445    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ffb72e-4672-4f8f-8904-37144c6897cd-tls-certs\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.524721 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.524509    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-home\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.524721 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.524548    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-model-cache\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.524721 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.524573    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-tmp-dir\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.524721 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.524598    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stgkh\" (UniqueName: \"kubernetes.io/projected/d0ffb72e-4672-4f8f-8904-37144c6897cd-kube-api-access-stgkh\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.524721 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.524685    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-dshm\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.525087 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.525051    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-home\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.525087 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.525064    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-tmp-dir\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.525266 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.525163    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-model-cache\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.526981 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.526956    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-dshm\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.527230 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.527212    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ffb72e-4672-4f8f-8904-37144c6897cd-tls-certs\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.532230 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.532203    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stgkh\" (UniqueName: \"kubernetes.io/projected/d0ffb72e-4672-4f8f-8904-37144c6897cd-kube-api-access-stgkh\") pod \"sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") " pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.610355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.610250    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:37.754155 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:37.754115    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"]
Jul 17 18:54:37.758721 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:54:37.758692    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0ffb72e_4672_4f8f_8904_37144c6897cd.slice/crio-e7a5c6a3abfe6678c8d20032c5c2461f8a50c7a298817c61e95940462b5ca1b1 WatchSource:0}: Error finding container e7a5c6a3abfe6678c8d20032c5c2461f8a50c7a298817c61e95940462b5ca1b1: Status 404 returned error can't find the container with id e7a5c6a3abfe6678c8d20032c5c2461f8a50c7a298817c61e95940462b5ca1b1
Jul 17 18:54:38.426974 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:38.426935    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2" event={"ID":"d0ffb72e-4672-4f8f-8904-37144c6897cd","Type":"ContainerStarted","Data":"51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516"}
Jul 17 18:54:38.426974 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:38.426986    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2" event={"ID":"d0ffb72e-4672-4f8f-8904-37144c6897cd","Type":"ContainerStarted","Data":"e7a5c6a3abfe6678c8d20032c5c2461f8a50c7a298817c61e95940462b5ca1b1"}
Jul 17 18:54:38.446959 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:38.446890    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2" podStartSLOduration=1.446872199 podStartE2EDuration="1.446872199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:54:38.443605007 +0000 UTC m=+3476.029081261" watchObservedRunningTime="2026-07-17 18:54:38.446872199 +0000 UTC m=+3476.032348517"
Jul 17 18:54:41.784962 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:41.784925    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:54:47.610384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:47.610346    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:47.610824 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:47.610404    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:47.623258 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:47.623225    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:48.502914 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:48.502886    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:54:59.600415 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.600370    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"]
Jul 17 18:54:59.604756 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.604730    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.608061 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.608032    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"kube-root-ca.crt\""
Jul 17 18:54:59.608280 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.608258    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"sub-test-model-52fc47-kserve-self-signed-certs\""
Jul 17 18:54:59.608454 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.608204    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"openshift-service-ca.crt\""
Jul 17 18:54:59.608610 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.608148    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"default-dockercfg-b9hbn\""
Jul 17 18:54:59.611128 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.611104    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"]
Jul 17 18:54:59.638284 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.638243    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-home\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.638284 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.638282    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-model-cache\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.638573 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.638371    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-dshm\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.638573 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.638408    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j68fw\" (UniqueName: \"kubernetes.io/projected/54c2e826-6543-440f-be65-7a9dcede0781-kube-api-access-j68fw\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.638573 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.638467    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54c2e826-6543-440f-be65-7a9dcede0781-tls-certs\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.638573 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.638510    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-tmp-dir\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.739797 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.739763    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-home\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.739797 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.739798    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-model-cache\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.740064 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.739819    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-dshm\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.740064 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.739836    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68fw\" (UniqueName: \"kubernetes.io/projected/54c2e826-6543-440f-be65-7a9dcede0781-kube-api-access-j68fw\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.740064 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.739873    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54c2e826-6543-440f-be65-7a9dcede0781-tls-certs\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.740582 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.739905    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-tmp-dir\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.741313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.740716    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-model-cache\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.741313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.740745    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-home\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.741313 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.741101    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-tmp-dir\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.751051 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.750994    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54c2e826-6543-440f-be65-7a9dcede0781-tls-certs\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.752866 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.752835    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-dshm\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.753064 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.753048    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j68fw\" (UniqueName: \"kubernetes.io/projected/54c2e826-6543-440f-be65-7a9dcede0781-kube-api-access-j68fw\") pod \"sub-test-model-52fc47-kserve-9d5cd697-xmgz2\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") " pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:54:59.917179 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:54:59.917089    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:55:00.070037 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:00.070008    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"]
Jul 17 18:55:00.071672 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:55:00.071623    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54c2e826_6543_440f_be65_7a9dcede0781.slice/crio-a6d082c5cfa1046f79ef6cdfaf21b9af7c6ebc21dc4e7f502164eb71095d6076 WatchSource:0}: Error finding container a6d082c5cfa1046f79ef6cdfaf21b9af7c6ebc21dc4e7f502164eb71095d6076: Status 404 returned error can't find the container with id a6d082c5cfa1046f79ef6cdfaf21b9af7c6ebc21dc4e7f502164eb71095d6076
Jul 17 18:55:00.566131 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:00.566091    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2" event={"ID":"54c2e826-6543-440f-be65-7a9dcede0781","Type":"ContainerStarted","Data":"17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223"}
Jul 17 18:55:00.566320 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:00.566141    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2" event={"ID":"54c2e826-6543-440f-be65-7a9dcede0781","Type":"ContainerStarted","Data":"a6d082c5cfa1046f79ef6cdfaf21b9af7c6ebc21dc4e7f502164eb71095d6076"}
Jul 17 18:55:00.583522 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:00.583463    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2" podStartSLOduration=1.5834476610000001 podStartE2EDuration="1.583447661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:00.581264541 +0000 UTC m=+3498.166740794" watchObservedRunningTime="2026-07-17 18:55:00.583447661 +0000 UTC m=+3498.168923913"
Jul 17 18:55:04.392920 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:04.392823    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:55:09.917499 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:09.917459    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:55:09.917949 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:09.917528    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:55:09.930271 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:09.930239    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:55:10.638429 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:10.638391    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:55:23.984632 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:23.984589    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:55:44.786093 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:44.786052    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:55:50.229508 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:50.229469    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"]
Jul 17 18:55:51.102954 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:55:51.102913    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"]
Jul 17 18:56:04.199016 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:04.198981    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:56:05.251168 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.251129    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2" podUID="d0ffb72e-4672-4f8f-8904-37144c6897cd" containerName="main" containerID="cri-o://51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516" gracePeriod=45
Jul 17 18:56:05.513344 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.513262    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:56:05.603966 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.603922    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-tmp-dir\") pod \"d0ffb72e-4672-4f8f-8904-37144c6897cd\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") "
Jul 17 18:56:05.604169 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.603985    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stgkh\" (UniqueName: \"kubernetes.io/projected/d0ffb72e-4672-4f8f-8904-37144c6897cd-kube-api-access-stgkh\") pod \"d0ffb72e-4672-4f8f-8904-37144c6897cd\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") "
Jul 17 18:56:05.604169 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604032    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-dshm\") pod \"d0ffb72e-4672-4f8f-8904-37144c6897cd\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") "
Jul 17 18:56:05.604169 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604055    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-home\") pod \"d0ffb72e-4672-4f8f-8904-37144c6897cd\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") "
Jul 17 18:56:05.604355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604198    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ffb72e-4672-4f8f-8904-37144c6897cd-tls-certs\") pod \"d0ffb72e-4672-4f8f-8904-37144c6897cd\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") "
Jul 17 18:56:05.604355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604245    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0ffb72e-4672-4f8f-8904-37144c6897cd" (UID: "d0ffb72e-4672-4f8f-8904-37144c6897cd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:05.604355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604260    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-model-cache\") pod \"d0ffb72e-4672-4f8f-8904-37144c6897cd\" (UID: \"d0ffb72e-4672-4f8f-8904-37144c6897cd\") "
Jul 17 18:56:05.604355 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604297    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-home" (OuterVolumeSpecName: "home") pod "d0ffb72e-4672-4f8f-8904-37144c6897cd" (UID: "d0ffb72e-4672-4f8f-8904-37144c6897cd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:05.604540 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604511    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0ffb72e-4672-4f8f-8904-37144c6897cd" (UID: "d0ffb72e-4672-4f8f-8904-37144c6897cd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:05.604672 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604636    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:05.604672 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604673    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:05.604843 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.604688    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:05.606269 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.606248    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-dshm" (OuterVolumeSpecName: "dshm") pod "d0ffb72e-4672-4f8f-8904-37144c6897cd" (UID: "d0ffb72e-4672-4f8f-8904-37144c6897cd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:05.606372 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.606279    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ffb72e-4672-4f8f-8904-37144c6897cd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0ffb72e-4672-4f8f-8904-37144c6897cd" (UID: "d0ffb72e-4672-4f8f-8904-37144c6897cd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:56:05.606372 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.606349    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ffb72e-4672-4f8f-8904-37144c6897cd-kube-api-access-stgkh" (OuterVolumeSpecName: "kube-api-access-stgkh") pod "d0ffb72e-4672-4f8f-8904-37144c6897cd" (UID: "d0ffb72e-4672-4f8f-8904-37144c6897cd"). InnerVolumeSpecName "kube-api-access-stgkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:56:05.705858 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.705814    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stgkh\" (UniqueName: \"kubernetes.io/projected/d0ffb72e-4672-4f8f-8904-37144c6897cd-kube-api-access-stgkh\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:05.705858 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.705852    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ffb72e-4672-4f8f-8904-37144c6897cd-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:05.705858 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.705861    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ffb72e-4672-4f8f-8904-37144c6897cd-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:05.967992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.967898    2555 generic.go:356] "Generic (PLEG): container finished" podID="d0ffb72e-4672-4f8f-8904-37144c6897cd" containerID="51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516" exitCode=0
Jul 17 18:56:05.967992 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.967947    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"
Jul 17 18:56:05.968181 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.967989    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2" event={"ID":"d0ffb72e-4672-4f8f-8904-37144c6897cd","Type":"ContainerDied","Data":"51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516"}
Jul 17 18:56:05.968181 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.968037    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2" event={"ID":"d0ffb72e-4672-4f8f-8904-37144c6897cd","Type":"ContainerDied","Data":"e7a5c6a3abfe6678c8d20032c5c2461f8a50c7a298817c61e95940462b5ca1b1"}
Jul 17 18:56:05.968181 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.968054    2555 scope.go:117] "RemoveContainer" containerID="51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516"
Jul 17 18:56:05.977961 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.977942    2555 scope.go:117] "RemoveContainer" containerID="51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516"
Jul 17 18:56:05.978239 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:56:05.978222    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516\": container with ID starting with 51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516 not found: ID does not exist" containerID="51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516"
Jul 17 18:56:05.978276 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.978248    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516"} err="failed to get container status \"51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516\": rpc error: code = NotFound desc = could not find container \"51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516\": container with ID starting with 51fffc49186447cb380ccd4b98a5f83727a6a128715afa8856930cc182d04516 not found: ID does not exist"
Jul 17 18:56:05.992012 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.991976    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"]
Jul 17 18:56:05.997582 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:05.997551    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/sub-test-model-7d6f90-kserve-85d8dff56c-9ktr2"]
Jul 17 18:56:06.123674 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.123604    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2" podUID="54c2e826-6543-440f-be65-7a9dcede0781" containerName="main" containerID="cri-o://17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223" gracePeriod=45
Jul 17 18:56:06.388745 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.388713    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:56:06.411133 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411097    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-model-cache\") pod \"54c2e826-6543-440f-be65-7a9dcede0781\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") "
Jul 17 18:56:06.411330 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411203    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54c2e826-6543-440f-be65-7a9dcede0781-tls-certs\") pod \"54c2e826-6543-440f-be65-7a9dcede0781\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") "
Jul 17 18:56:06.411330 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411267    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-dshm\") pod \"54c2e826-6543-440f-be65-7a9dcede0781\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") "
Jul 17 18:56:06.411330 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411296    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-home\") pod \"54c2e826-6543-440f-be65-7a9dcede0781\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") "
Jul 17 18:56:06.411495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411334    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j68fw\" (UniqueName: \"kubernetes.io/projected/54c2e826-6543-440f-be65-7a9dcede0781-kube-api-access-j68fw\") pod \"54c2e826-6543-440f-be65-7a9dcede0781\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") "
Jul 17 18:56:06.411495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411359    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-tmp-dir\") pod \"54c2e826-6543-440f-be65-7a9dcede0781\" (UID: \"54c2e826-6543-440f-be65-7a9dcede0781\") "
Jul 17 18:56:06.411495 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411425    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-model-cache" (OuterVolumeSpecName: "model-cache") pod "54c2e826-6543-440f-be65-7a9dcede0781" (UID: "54c2e826-6543-440f-be65-7a9dcede0781"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:06.411690 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411603    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-home" (OuterVolumeSpecName: "home") pod "54c2e826-6543-440f-be65-7a9dcede0781" (UID: "54c2e826-6543-440f-be65-7a9dcede0781"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:06.411690 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411663    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "54c2e826-6543-440f-be65-7a9dcede0781" (UID: "54c2e826-6543-440f-be65-7a9dcede0781"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:06.411832 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411774    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:06.411832 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411789    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:06.411832 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.411801    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:06.413581 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.413546    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-dshm" (OuterVolumeSpecName: "dshm") pod "54c2e826-6543-440f-be65-7a9dcede0781" (UID: "54c2e826-6543-440f-be65-7a9dcede0781"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:56:06.413723 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.413579    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54c2e826-6543-440f-be65-7a9dcede0781-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "54c2e826-6543-440f-be65-7a9dcede0781" (UID: "54c2e826-6543-440f-be65-7a9dcede0781"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:56:06.413825 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.413801    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c2e826-6543-440f-be65-7a9dcede0781-kube-api-access-j68fw" (OuterVolumeSpecName: "kube-api-access-j68fw") pod "54c2e826-6543-440f-be65-7a9dcede0781" (UID: "54c2e826-6543-440f-be65-7a9dcede0781"). InnerVolumeSpecName "kube-api-access-j68fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:56:06.512199 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.512155    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54c2e826-6543-440f-be65-7a9dcede0781-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:06.512199 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.512192    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54c2e826-6543-440f-be65-7a9dcede0781-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:06.512199 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.512201    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j68fw\" (UniqueName: \"kubernetes.io/projected/54c2e826-6543-440f-be65-7a9dcede0781-kube-api-access-j68fw\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:56:06.944993 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.944910    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ffb72e-4672-4f8f-8904-37144c6897cd" path="/var/lib/kubelet/pods/d0ffb72e-4672-4f8f-8904-37144c6897cd/volumes"
Jul 17 18:56:06.976079 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.976046    2555 generic.go:356] "Generic (PLEG): container finished" podID="54c2e826-6543-440f-be65-7a9dcede0781" containerID="17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223" exitCode=0
Jul 17 18:56:06.976262 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.976088    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"
Jul 17 18:56:06.976262 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.976128    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2" event={"ID":"54c2e826-6543-440f-be65-7a9dcede0781","Type":"ContainerDied","Data":"17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223"}
Jul 17 18:56:06.976262 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.976170    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2" event={"ID":"54c2e826-6543-440f-be65-7a9dcede0781","Type":"ContainerDied","Data":"a6d082c5cfa1046f79ef6cdfaf21b9af7c6ebc21dc4e7f502164eb71095d6076"}
Jul 17 18:56:06.976262 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.976189    2555 scope.go:117] "RemoveContainer" containerID="17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223"
Jul 17 18:56:06.985959 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.985939    2555 scope.go:117] "RemoveContainer" containerID="17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223"
Jul 17 18:56:06.986261 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:56:06.986242    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223\": container with ID starting with 17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223 not found: ID does not exist" containerID="17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223"
Jul 17 18:56:06.986301 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.986276    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223"} err="failed to get container status \"17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223\": rpc error: code = NotFound desc = could not find container \"17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223\": container with ID starting with 17a93c18281df1c37a6d061a92cdfa07527024a4f1c9203a893f87b04182d223 not found: ID does not exist"
Jul 17 18:56:06.993049 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.993006    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"]
Jul 17 18:56:06.996288 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:06.996258    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/sub-test-model-52fc47-kserve-9d5cd697-xmgz2"]
Jul 17 18:56:08.944606 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:08.944569    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54c2e826-6543-440f-be65-7a9dcede0781" path="/var/lib/kubelet/pods/54c2e826-6543-440f-be65-7a9dcede0781/volumes"
Jul 17 18:56:09.276127 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.276091    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"]
Jul 17 18:56:09.276790 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.276772    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c2e826-6543-440f-be65-7a9dcede0781" containerName="main"
Jul 17 18:56:09.276844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.276793    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c2e826-6543-440f-be65-7a9dcede0781" containerName="main"
Jul 17 18:56:09.276844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.276812    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ffb72e-4672-4f8f-8904-37144c6897cd" containerName="main"
Jul 17 18:56:09.276844 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.276822    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ffb72e-4672-4f8f-8904-37144c6897cd" containerName="main"
Jul 17 18:56:09.276954 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.276943    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ffb72e-4672-4f8f-8904-37144c6897cd" containerName="main"
Jul 17 18:56:09.276991 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.276959    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c2e826-6543-440f-be65-7a9dcede0781" containerName="main"
Jul 17 18:56:09.281726 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.281703    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.286509 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.285665    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"openshift-service-ca.crt\""
Jul 17 18:56:09.286509 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.285818    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"kube-root-ca.crt\""
Jul 17 18:56:09.286509 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.285818    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"default-dockercfg-zlqbb\""
Jul 17 18:56:09.286509 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.285933    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7d6f90\"/\"rate-test-model-7d6f90-kserve-self-signed-certs\""
Jul 17 18:56:09.289577 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.289548    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"]
Jul 17 18:56:09.336886 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.336848    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b3af8f6-5d6d-4066-a314-ad3eed860820-tls-certs\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.337097 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.336893    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-tmp-dir\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.337097 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.336933    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-model-cache\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.337097 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.337051    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-dshm\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.337097 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.337077    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-home\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.337273 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.337100    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z5dx\" (UniqueName: \"kubernetes.io/projected/5b3af8f6-5d6d-4066-a314-ad3eed860820-kube-api-access-8z5dx\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.438604 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.438556    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-dshm\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.438604 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.438602    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-home\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.438884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.438627    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z5dx\" (UniqueName: \"kubernetes.io/projected/5b3af8f6-5d6d-4066-a314-ad3eed860820-kube-api-access-8z5dx\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.438884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.438691    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b3af8f6-5d6d-4066-a314-ad3eed860820-tls-certs\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.438884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.438721    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-tmp-dir\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.438884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.438760    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-model-cache\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.439158 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.439133    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-home\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.439230 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.439194    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-tmp-dir\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.439424 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.439402    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-model-cache\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.441225 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.441198    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-dshm\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.441525 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.441510    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b3af8f6-5d6d-4066-a314-ad3eed860820-tls-certs\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.448023 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.447986    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z5dx\" (UniqueName: \"kubernetes.io/projected/5b3af8f6-5d6d-4066-a314-ad3eed860820-kube-api-access-8z5dx\") pod \"rate-test-model-7d6f90-kserve-7487b45667-hcpxg\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") " pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.594413 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.594318    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:09.742393 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.742326    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"]
Jul 17 18:56:09.744770 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:56:09.744743    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b3af8f6_5d6d_4066_a314_ad3eed860820.slice/crio-dc9e9b11388a16141bd02e408e5ffe6750e0bfa61f923dede4880f934de9103b WatchSource:0}: Error finding container dc9e9b11388a16141bd02e408e5ffe6750e0bfa61f923dede4880f934de9103b: Status 404 returned error can't find the container with id dc9e9b11388a16141bd02e408e5ffe6750e0bfa61f923dede4880f934de9103b
Jul 17 18:56:09.997410 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.997375    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg" event={"ID":"5b3af8f6-5d6d-4066-a314-ad3eed860820","Type":"ContainerStarted","Data":"6a114b6c474163119af5abe6237c3ba12ba8746f36a6af32b83250543ee754c8"}
Jul 17 18:56:09.997839 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:09.997425    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg" event={"ID":"5b3af8f6-5d6d-4066-a314-ad3eed860820","Type":"ContainerStarted","Data":"dc9e9b11388a16141bd02e408e5ffe6750e0bfa61f923dede4880f934de9103b"}
Jul 17 18:56:10.014681 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:10.014608    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg" podStartSLOduration=1.014591668 podStartE2EDuration="1.014591668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:10.013048551 +0000 UTC m=+3567.598524804" watchObservedRunningTime="2026-07-17 18:56:10.014591668 +0000 UTC m=+3567.600067919"
Jul 17 18:56:16.403055 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:16.403014    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:56:19.594563 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:19.594522    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:19.595033 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:19.594582    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:19.606971 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:19.606941    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:20.069917 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:20.069885    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:56:34.998795 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:34.998714    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"]
Jul 17 18:56:35.002692 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.002666    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.006339 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.006285    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"openshift-service-ca.crt\""
Jul 17 18:56:35.006522 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.006507    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"rate-test-model-52fc47-kserve-self-signed-certs\""
Jul 17 18:56:35.006612 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.006596    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"kube-root-ca.crt\""
Jul 17 18:56:35.006704 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.006628    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52fc47\"/\"default-dockercfg-b9hbn\""
Jul 17 18:56:35.013231 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.013198    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"]
Jul 17 18:56:35.098954 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.098918    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgb8x\" (UniqueName: \"kubernetes.io/projected/f10c37e6-e322-4ac3-a4de-bd919017f9d6-kube-api-access-hgb8x\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.099146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.098985    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-home\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.099146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.099012    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tmp-dir\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.099146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.099038    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-dshm\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.099146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.099053    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-model-cache\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.099146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.099117    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tls-certs\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.200636 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.200597    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgb8x\" (UniqueName: \"kubernetes.io/projected/f10c37e6-e322-4ac3-a4de-bd919017f9d6-kube-api-access-hgb8x\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.200884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.200724    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-home\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.200884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.200761    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tmp-dir\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.200884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.200794    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-dshm\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.200884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.200814    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-model-cache\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.200884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.200842    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tls-certs\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.201212 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.201184    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-home\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.201287 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.201249    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tmp-dir\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.201287 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.201268    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-model-cache\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.203205 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.203173    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-dshm\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.203416 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.203395    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tls-certs\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.209563 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.209541    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgb8x\" (UniqueName: \"kubernetes.io/projected/f10c37e6-e322-4ac3-a4de-bd919017f9d6-kube-api-access-hgb8x\") pod \"rate-test-model-52fc47-kserve-77f6597955-xwf4h\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") " pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.314159 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.314067    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:35.460888 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:35.460850    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"]
Jul 17 18:56:35.462429 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:56:35.462402    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf10c37e6_e322_4ac3_a4de_bd919017f9d6.slice/crio-dd8c174de0f3ec848a8f2019a0e402791ffec3f0c66b285dd820ff921c9ddecd WatchSource:0}: Error finding container dd8c174de0f3ec848a8f2019a0e402791ffec3f0c66b285dd820ff921c9ddecd: Status 404 returned error can't find the container with id dd8c174de0f3ec848a8f2019a0e402791ffec3f0c66b285dd820ff921c9ddecd
Jul 17 18:56:36.158687 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:36.158626    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h" event={"ID":"f10c37e6-e322-4ac3-a4de-bd919017f9d6","Type":"ContainerStarted","Data":"7635836e1af3c1a7182b05f851520ae497da4e5803968f35e877704edf1efe21"}
Jul 17 18:56:36.159091 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:36.158698    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h" event={"ID":"f10c37e6-e322-4ac3-a4de-bd919017f9d6","Type":"ContainerStarted","Data":"dd8c174de0f3ec848a8f2019a0e402791ffec3f0c66b285dd820ff921c9ddecd"}
Jul 17 18:56:36.179964 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:36.179902    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h" podStartSLOduration=2.179886688 podStartE2EDuration="2.179886688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:36.177102787 +0000 UTC m=+3593.762579043" watchObservedRunningTime="2026-07-17 18:56:36.179886688 +0000 UTC m=+3593.765362996"
Jul 17 18:56:40.088838 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:40.088800    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:56:45.314948 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:45.314891    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:45.315384 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:45.314962    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:45.327570 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:45.327537    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:56:46.234237 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:56:46.234203    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:57:06.496522 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:06.496484    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:57:25.320426 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:25.320386    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:57:30.385259 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:30.385216    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"]
Jul 17 18:57:31.365635 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:31.365591    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"]
Jul 17 18:57:43.426418 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:43.426385    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:57:45.405351 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.405308    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg" podUID="5b3af8f6-5d6d-4066-a314-ad3eed860820" containerName="main" containerID="cri-o://6a114b6c474163119af5abe6237c3ba12ba8746f36a6af32b83250543ee754c8" gracePeriod=45
Jul 17 18:57:45.592015 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.591971    2555 generic.go:356] "Generic (PLEG): container finished" podID="5b3af8f6-5d6d-4066-a314-ad3eed860820" containerID="6a114b6c474163119af5abe6237c3ba12ba8746f36a6af32b83250543ee754c8" exitCode=0
Jul 17 18:57:45.592176 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.592038    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg" event={"ID":"5b3af8f6-5d6d-4066-a314-ad3eed860820","Type":"ContainerDied","Data":"6a114b6c474163119af5abe6237c3ba12ba8746f36a6af32b83250543ee754c8"}
Jul 17 18:57:45.671064 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.670998    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:57:45.778336 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778304    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-model-cache\") pod \"5b3af8f6-5d6d-4066-a314-ad3eed860820\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") "
Jul 17 18:57:45.778541 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778345    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-home\") pod \"5b3af8f6-5d6d-4066-a314-ad3eed860820\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") "
Jul 17 18:57:45.778541 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778406    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-tmp-dir\") pod \"5b3af8f6-5d6d-4066-a314-ad3eed860820\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") "
Jul 17 18:57:45.778541 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778423    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-dshm\") pod \"5b3af8f6-5d6d-4066-a314-ad3eed860820\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") "
Jul 17 18:57:45.778541 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778442    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b3af8f6-5d6d-4066-a314-ad3eed860820-tls-certs\") pod \"5b3af8f6-5d6d-4066-a314-ad3eed860820\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") "
Jul 17 18:57:45.778541 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778493    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z5dx\" (UniqueName: \"kubernetes.io/projected/5b3af8f6-5d6d-4066-a314-ad3eed860820-kube-api-access-8z5dx\") pod \"5b3af8f6-5d6d-4066-a314-ad3eed860820\" (UID: \"5b3af8f6-5d6d-4066-a314-ad3eed860820\") "
Jul 17 18:57:45.779056 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778710    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b3af8f6-5d6d-4066-a314-ad3eed860820" (UID: "5b3af8f6-5d6d-4066-a314-ad3eed860820"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:45.779056 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778755    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b3af8f6-5d6d-4066-a314-ad3eed860820" (UID: "5b3af8f6-5d6d-4066-a314-ad3eed860820"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:45.779056 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.778783    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-home" (OuterVolumeSpecName: "home") pod "5b3af8f6-5d6d-4066-a314-ad3eed860820" (UID: "5b3af8f6-5d6d-4066-a314-ad3eed860820"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:45.779056 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.779039    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:45.779206 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.779059    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:45.779206 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.779071    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:45.780902 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.780879    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-dshm" (OuterVolumeSpecName: "dshm") pod "5b3af8f6-5d6d-4066-a314-ad3eed860820" (UID: "5b3af8f6-5d6d-4066-a314-ad3eed860820"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:45.781128 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.781108    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b3af8f6-5d6d-4066-a314-ad3eed860820-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b3af8f6-5d6d-4066-a314-ad3eed860820" (UID: "5b3af8f6-5d6d-4066-a314-ad3eed860820"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:57:45.781175 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.781128    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b3af8f6-5d6d-4066-a314-ad3eed860820-kube-api-access-8z5dx" (OuterVolumeSpecName: "kube-api-access-8z5dx") pod "5b3af8f6-5d6d-4066-a314-ad3eed860820" (UID: "5b3af8f6-5d6d-4066-a314-ad3eed860820"). InnerVolumeSpecName "kube-api-access-8z5dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:57:45.879748 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.879711    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b3af8f6-5d6d-4066-a314-ad3eed860820-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:45.879748 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.879742    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b3af8f6-5d6d-4066-a314-ad3eed860820-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:45.879748 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:45.879754    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z5dx\" (UniqueName: \"kubernetes.io/projected/5b3af8f6-5d6d-4066-a314-ad3eed860820-kube-api-access-8z5dx\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:46.386265 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.386201    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h" podUID="f10c37e6-e322-4ac3-a4de-bd919017f9d6" containerName="main" containerID="cri-o://7635836e1af3c1a7182b05f851520ae497da4e5803968f35e877704edf1efe21" gracePeriod=45
Jul 17 18:57:46.600255 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.600215    2555 generic.go:356] "Generic (PLEG): container finished" podID="f10c37e6-e322-4ac3-a4de-bd919017f9d6" containerID="7635836e1af3c1a7182b05f851520ae497da4e5803968f35e877704edf1efe21" exitCode=0
Jul 17 18:57:46.600703 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.600282    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h" event={"ID":"f10c37e6-e322-4ac3-a4de-bd919017f9d6","Type":"ContainerDied","Data":"7635836e1af3c1a7182b05f851520ae497da4e5803968f35e877704edf1efe21"}
Jul 17 18:57:46.602140 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.602116    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"
Jul 17 18:57:46.602140 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.602127    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg" event={"ID":"5b3af8f6-5d6d-4066-a314-ad3eed860820","Type":"ContainerDied","Data":"dc9e9b11388a16141bd02e408e5ffe6750e0bfa61f923dede4880f934de9103b"}
Jul 17 18:57:46.602310 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.602180    2555 scope.go:117] "RemoveContainer" containerID="6a114b6c474163119af5abe6237c3ba12ba8746f36a6af32b83250543ee754c8"
Jul 17 18:57:46.636108 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.636076    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"]
Jul 17 18:57:46.641511 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.641480    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7d6f90/rate-test-model-7d6f90-kserve-7487b45667-hcpxg"]
Jul 17 18:57:46.663515 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.663485    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:57:46.786617 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.786579    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-model-cache\") pod \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") "
Jul 17 18:57:46.786848 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.786631    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgb8x\" (UniqueName: \"kubernetes.io/projected/f10c37e6-e322-4ac3-a4de-bd919017f9d6-kube-api-access-hgb8x\") pod \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") "
Jul 17 18:57:46.786848 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.786751    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tmp-dir\") pod \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") "
Jul 17 18:57:46.786848 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.786771    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-home\") pod \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") "
Jul 17 18:57:46.787024 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.786859    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-dshm\") pod \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") "
Jul 17 18:57:46.787024 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.786895    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tls-certs\") pod \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\" (UID: \"f10c37e6-e322-4ac3-a4de-bd919017f9d6\") "
Jul 17 18:57:46.787024 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.786945    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "f10c37e6-e322-4ac3-a4de-bd919017f9d6" (UID: "f10c37e6-e322-4ac3-a4de-bd919017f9d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:46.787155 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.787031    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-home" (OuterVolumeSpecName: "home") pod "f10c37e6-e322-4ac3-a4de-bd919017f9d6" (UID: "f10c37e6-e322-4ac3-a4de-bd919017f9d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:46.787155 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.787046    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f10c37e6-e322-4ac3-a4de-bd919017f9d6" (UID: "f10c37e6-e322-4ac3-a4de-bd919017f9d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:46.787258 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.787238    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:46.787301 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.787263    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:46.787301 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.787277    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:46.788945 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.788922    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f10c37e6-e322-4ac3-a4de-bd919017f9d6" (UID: "f10c37e6-e322-4ac3-a4de-bd919017f9d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 18:57:46.789028 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.789001    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-dshm" (OuterVolumeSpecName: "dshm") pod "f10c37e6-e322-4ac3-a4de-bd919017f9d6" (UID: "f10c37e6-e322-4ac3-a4de-bd919017f9d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 18:57:46.789028 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.789015    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f10c37e6-e322-4ac3-a4de-bd919017f9d6-kube-api-access-hgb8x" (OuterVolumeSpecName: "kube-api-access-hgb8x") pod "f10c37e6-e322-4ac3-a4de-bd919017f9d6" (UID: "f10c37e6-e322-4ac3-a4de-bd919017f9d6"). InnerVolumeSpecName "kube-api-access-hgb8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:57:46.888571 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.888528    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10c37e6-e322-4ac3-a4de-bd919017f9d6-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:46.888571 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.888564    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgb8x\" (UniqueName: \"kubernetes.io/projected/f10c37e6-e322-4ac3-a4de-bd919017f9d6-kube-api-access-hgb8x\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:46.888571 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.888575    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10c37e6-e322-4ac3-a4de-bd919017f9d6-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:46.945131 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:46.945043    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b3af8f6-5d6d-4066-a314-ad3eed860820" path="/var/lib/kubelet/pods/5b3af8f6-5d6d-4066-a314-ad3eed860820/volumes"
Jul 17 18:57:47.014788 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.014755    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7278-139673085021472"]
Jul 17 18:57:47.015194 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.015182    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b3af8f6-5d6d-4066-a314-ad3eed860820" containerName="main"
Jul 17 18:57:47.015253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.015196    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b3af8f6-5d6d-4066-a314-ad3eed860820" containerName="main"
Jul 17 18:57:47.015253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.015205    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f10c37e6-e322-4ac3-a4de-bd919017f9d6" containerName="main"
Jul 17 18:57:47.015253 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.015210    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f10c37e6-e322-4ac3-a4de-bd919017f9d6" containerName="main"
Jul 17 18:57:47.015351 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.015291    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f10c37e6-e322-4ac3-a4de-bd919017f9d6" containerName="main"
Jul 17 18:57:47.015351 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.015301    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b3af8f6-5d6d-4066-a314-ad3eed860820" containerName="main"
Jul 17 18:57:47.018728 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.018711    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:47.021822 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.021799    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-f4nxw\""
Jul 17 18:57:47.031765 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.031734    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7278-139673085021472"]
Jul 17 18:57:47.090994 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.090954    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw85z\" (UniqueName: \"kubernetes.io/projected/47b26523-1872-4b8c-a5c3-7573ce65cb55-kube-api-access-qw85z\") pod \"test-curl-7278-139673085021472\" (UID: \"47b26523-1872-4b8c-a5c3-7573ce65cb55\") " pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:47.192309 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.192268    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw85z\" (UniqueName: \"kubernetes.io/projected/47b26523-1872-4b8c-a5c3-7573ce65cb55-kube-api-access-qw85z\") pod \"test-curl-7278-139673085021472\" (UID: \"47b26523-1872-4b8c-a5c3-7573ce65cb55\") " pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:47.203941 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.203872    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw85z\" (UniqueName: \"kubernetes.io/projected/47b26523-1872-4b8c-a5c3-7573ce65cb55-kube-api-access-qw85z\") pod \"test-curl-7278-139673085021472\" (UID: \"47b26523-1872-4b8c-a5c3-7573ce65cb55\") " pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:47.328513 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.328472    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:47.504924 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.504889    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7278-139673085021472"]
Jul 17 18:57:47.506531 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:57:47.506502    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47b26523_1872_4b8c_a5c3_7573ce65cb55.slice/crio-314768ac0054e75e2282e4a9738daf54147f43af1818ffb74008075e6cc9e4ce WatchSource:0}: Error finding container 314768ac0054e75e2282e4a9738daf54147f43af1818ffb74008075e6cc9e4ce: Status 404 returned error can't find the container with id 314768ac0054e75e2282e4a9738daf54147f43af1818ffb74008075e6cc9e4ce
Jul 17 18:57:47.507952 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.507935    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 18:57:47.610211 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.610177    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7278-139673085021472" event={"ID":"47b26523-1872-4b8c-a5c3-7573ce65cb55","Type":"ContainerStarted","Data":"314768ac0054e75e2282e4a9738daf54147f43af1818ffb74008075e6cc9e4ce"}
Jul 17 18:57:47.611655 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.611615    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h" event={"ID":"f10c37e6-e322-4ac3-a4de-bd919017f9d6","Type":"ContainerDied","Data":"dd8c174de0f3ec848a8f2019a0e402791ffec3f0c66b285dd820ff921c9ddecd"}
Jul 17 18:57:47.611790 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.611690    2555 scope.go:117] "RemoveContainer" containerID="7635836e1af3c1a7182b05f851520ae497da4e5803968f35e877704edf1efe21"
Jul 17 18:57:47.611790 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.611623    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"
Jul 17 18:57:47.641797 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.641751    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"]
Jul 17 18:57:47.651244 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:47.651214    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-52fc47/rate-test-model-52fc47-kserve-77f6597955-xwf4h"]
Jul 17 18:57:48.621848 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:48.621751    2555 generic.go:356] "Generic (PLEG): container finished" podID="47b26523-1872-4b8c-a5c3-7573ce65cb55" containerID="4c7c614e8db48dcdf39f48f03c16ad9aab99f5ad26ceeb235ba92c8a0f96af80" exitCode=0
Jul 17 18:57:48.622254 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:48.621841    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7278-139673085021472" event={"ID":"47b26523-1872-4b8c-a5c3-7573ce65cb55","Type":"ContainerDied","Data":"4c7c614e8db48dcdf39f48f03c16ad9aab99f5ad26ceeb235ba92c8a0f96af80"}
Jul 17 18:57:48.944615 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:48.944512    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f10c37e6-e322-4ac3-a4de-bd919017f9d6" path="/var/lib/kubelet/pods/f10c37e6-e322-4ac3-a4de-bd919017f9d6/volumes"
Jul 17 18:57:49.783247 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:49.783212    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:49.830386 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:49.830351    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7278-139673085021472_47b26523-1872-4b8c-a5c3-7573ce65cb55/test-curl-7278-139673085021472/0.log"
Jul 17 18:57:49.858134 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:49.858089    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7278-139673085021472"]
Jul 17 18:57:49.863305 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:49.863268    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7278-139673085021472"]
Jul 17 18:57:49.920671 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:49.920564    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw85z\" (UniqueName: \"kubernetes.io/projected/47b26523-1872-4b8c-a5c3-7573ce65cb55-kube-api-access-qw85z\") pod \"47b26523-1872-4b8c-a5c3-7573ce65cb55\" (UID: \"47b26523-1872-4b8c-a5c3-7573ce65cb55\") "
Jul 17 18:57:49.922732 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:49.922698    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b26523-1872-4b8c-a5c3-7573ce65cb55-kube-api-access-qw85z" (OuterVolumeSpecName: "kube-api-access-qw85z") pod "47b26523-1872-4b8c-a5c3-7573ce65cb55" (UID: "47b26523-1872-4b8c-a5c3-7573ce65cb55"). InnerVolumeSpecName "kube-api-access-qw85z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 18:57:50.022020 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:50.021978    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw85z\" (UniqueName: \"kubernetes.io/projected/47b26523-1872-4b8c-a5c3-7573ce65cb55-kube-api-access-qw85z\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Jul 17 18:57:50.635724 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:50.635684    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:50.635884 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:50.635702    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="314768ac0054e75e2282e4a9738daf54147f43af1818ffb74008075e6cc9e4ce"
Jul 17 18:57:50.648849 ip-10-0-140-23 kubenswrapper[2555]: E0717 18:57:50.648817    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7278-139673085021472\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="47b26523-1872-4b8c-a5c3-7573ce65cb55" pod="opendatahub/test-curl-7278-139673085021472"
Jul 17 18:57:50.946085 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:57:50.945984    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47b26523-1872-4b8c-a5c3-7573ce65cb55" path="/var/lib/kubelet/pods/47b26523-1872-4b8c-a5c3-7573ce65cb55/volumes"
Jul 17 18:58:04.853449 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:04.853277    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:58:31.335891 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:31.335847    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cbfck"]
Jul 17 18:58:43.497454 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:43.497418    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bdd7ff788-zsqkf_da2eb1e6-14d0-4a3f-b9e3-ec647df07559/authorino/0.log"
Jul 17 18:58:49.772819 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:49.772784    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-fb5cz_1e7f6181-4f4b-4ef7-a269-b847f43ba563/manager/0.log"
Jul 17 18:58:49.913761 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:49.913731    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cpxwg_2cb9e2ac-f565-44c3-a832-4a81a349cc4c/manager/0.log"
Jul 17 18:58:50.221957 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:50.221872    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qwmw7_02eb8a4d-4c18-4663-88ec-a19d0fab22a7/server/0.log"
Jul 17 18:58:50.363610 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:50.363565    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dr8fl_1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4/manager/0.log"
Jul 17 18:58:50.659605 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:50.659576    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d8bdcf84-72fkb_047e5ea0-35dd-4779-b821-27e1234abbcc/copy-manifests/0.log"
Jul 17 18:58:50.684828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:50.684800    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d8bdcf84-72fkb_047e5ea0-35dd-4779-b821-27e1234abbcc/manager/0.log"
Jul 17 18:58:51.337590 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:51.337554    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738535-25qfv_248a1faa-0782-45cc-8401-c751cfc42780/cleanup/0.log"
Jul 17 18:58:53.709839 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:53.709808    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr_e15b2e38-8b78-4a5a-85d1-cce9e80a793f/pull/0.log"
Jul 17 18:58:53.725059 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:53.725027    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr_e15b2e38-8b78-4a5a-85d1-cce9e80a793f/extract/0.log"
Jul 17 18:58:53.732740 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:53.732712    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr_e15b2e38-8b78-4a5a-85d1-cce9e80a793f/util/0.log"
Jul 17 18:58:53.861951 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:53.861922    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x_a25df112-ab04-4732-a67d-c1de4bf0b6af/util/0.log"
Jul 17 18:58:53.873074 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:53.873048    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x_a25df112-ab04-4732-a67d-c1de4bf0b6af/pull/0.log"
Jul 17 18:58:53.883011 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:53.882975    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x_a25df112-ab04-4732-a67d-c1de4bf0b6af/extract/0.log"
Jul 17 18:58:54.013286 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.013254    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46_9cf51f45-7174-441c-a45f-99db7a138df2/util/0.log"
Jul 17 18:58:54.026878 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.026847    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46_9cf51f45-7174-441c-a45f-99db7a138df2/pull/0.log"
Jul 17 18:58:54.035945 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.035918    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46_9cf51f45-7174-441c-a45f-99db7a138df2/extract/0.log"
Jul 17 18:58:54.160589 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.160558    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq_fb995d59-07b9-4ac8-ade4-db7e3fbfd59f/util/0.log"
Jul 17 18:58:54.173339 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.173311    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq_fb995d59-07b9-4ac8-ade4-db7e3fbfd59f/pull/0.log"
Jul 17 18:58:54.185222 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.185199    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq_fb995d59-07b9-4ac8-ade4-db7e3fbfd59f/extract/0.log"
Jul 17 18:58:54.345744 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.345635    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bdd7ff788-zsqkf_da2eb1e6-14d0-4a3f-b9e3-ec647df07559/authorino/0.log"
Jul 17 18:58:54.494966 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.494933    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-22gvf_4595cf40-0d56-4b29-902b-2a74306cd93c/manager/0.log"
Jul 17 18:58:54.623778 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.623695    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m6fdq_b929e9a6-fb69-4ce2-9605-20b21c4f9515/manager/0.log"
Jul 17 18:58:54.751961 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.751931    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xnbfk_fc78a7d7-9035-4cd4-b12e-56642e234855/kuadrant-console-plugin/0.log"
Jul 17 18:58:54.882117 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:54.882030    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mqfcm_2c4fda1f-ca78-4ff9-a735-de250a40e47f/registry-server/0.log"
Jul 17 18:58:55.045358 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:55.045323    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-g4dpw_f02c96ff-7c24-42c0-8937-49d85a643ff7/manager/0.log"
Jul 17 18:58:55.208000 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:55.207922    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cbfck_72209bda-6da3-4cd6-a0f0-6597c8628036/limitador/0.log"
Jul 17 18:58:55.369337 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:55.369306    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2rjpk_8295f3fd-3c4f-4cdd-8685-087c250085b4/manager/0.log"
Jul 17 18:58:56.027066 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:56.027040    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bfff449b-qmvlc_aa1a105c-005e-47b6-8093-036767bd1b35/kube-auth-proxy/0.log"
Jul 17 18:58:56.430699 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:56.430591    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zkwv7_3ace2039-18db-4d7b-90b0-303fbf522b6c/payload-pre-processing/0.log"
Jul 17 18:58:57.196138 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:57.196097    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zth9g_577243a4-ad30-408e-9e50-348b806771f2/main/0.log"
Jul 17 18:58:57.327344 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:57.327313    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-46n6s_5484266e-fb83-4cf4-99d2-46c331f78aba/main/0.log"
Jul 17 18:58:57.727549 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:58:57.727520    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-r7t2n_b90fd575-54c4-4dd8-ac57-7d804b53a37d/main/0.log"
Jul 17 18:59:08.255068 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:08.255033    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-snwmp_0c868384-1df1-438e-8e7c-6a7d578a58af/global-pull-secret-syncer/0.log"
Jul 17 18:59:08.554146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:08.554043    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wbb86_61fd3cf4-956c-45d3-a926-c0b39369e21c/konnectivity-agent/0.log"
Jul 17 18:59:08.796901 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:08.796857    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-23.ec2.internal_e9e7f913e9ca157cda0a9d37dab0aefd/haproxy/0.log"
Jul 17 18:59:12.690538 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:12.690505    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr_e15b2e38-8b78-4a5a-85d1-cce9e80a793f/extract/0.log"
Jul 17 18:59:12.758976 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:12.758938    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr_e15b2e38-8b78-4a5a-85d1-cce9e80a793f/util/0.log"
Jul 17 18:59:12.832073 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:12.832033    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fpdjr_e15b2e38-8b78-4a5a-85d1-cce9e80a793f/pull/0.log"
Jul 17 18:59:12.891578 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:12.891549    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x_a25df112-ab04-4732-a67d-c1de4bf0b6af/extract/0.log"
Jul 17 18:59:12.936508 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:12.936479    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x_a25df112-ab04-4732-a67d-c1de4bf0b6af/util/0.log"
Jul 17 18:59:12.992283 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:12.992252    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06xh4x_a25df112-ab04-4732-a67d-c1de4bf0b6af/pull/0.log"
Jul 17 18:59:13.041566 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:13.041537    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46_9cf51f45-7174-441c-a45f-99db7a138df2/extract/0.log"
Jul 17 18:59:13.098737 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:13.098711    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46_9cf51f45-7174-441c-a45f-99db7a138df2/util/0.log"
Jul 17 18:59:13.161761 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:13.161729    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73kqw46_9cf51f45-7174-441c-a45f-99db7a138df2/pull/0.log"
Jul 17 18:59:13.234041 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:13.234015    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq_fb995d59-07b9-4ac8-ade4-db7e3fbfd59f/extract/0.log"
Jul 17 18:59:13.346616 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:13.346525    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq_fb995d59-07b9-4ac8-ade4-db7e3fbfd59f/util/0.log"
Jul 17 18:59:13.389478 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:13.389447    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xpvtq_fb995d59-07b9-4ac8-ade4-db7e3fbfd59f/pull/0.log"
Jul 17 18:59:13.886356 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:13.886319    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bdd7ff788-zsqkf_da2eb1e6-14d0-4a3f-b9e3-ec647df07559/authorino/0.log"
Jul 17 18:59:14.047302 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:14.047265    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-22gvf_4595cf40-0d56-4b29-902b-2a74306cd93c/manager/0.log"
Jul 17 18:59:14.112793 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:14.112766    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m6fdq_b929e9a6-fb69-4ce2-9605-20b21c4f9515/manager/0.log"
Jul 17 18:59:14.174212 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:14.174123    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xnbfk_fc78a7d7-9035-4cd4-b12e-56642e234855/kuadrant-console-plugin/0.log"
Jul 17 18:59:14.273619 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:14.273587    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mqfcm_2c4fda1f-ca78-4ff9-a735-de250a40e47f/registry-server/0.log"
Jul 17 18:59:14.527492 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:14.527464    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-g4dpw_f02c96ff-7c24-42c0-8937-49d85a643ff7/manager/0.log"
Jul 17 18:59:14.663400 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:14.663366    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cbfck_72209bda-6da3-4cd6-a0f0-6597c8628036/limitador/0.log"
Jul 17 18:59:14.848317 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:14.848221    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2rjpk_8295f3fd-3c4f-4cdd-8685-087c250085b4/manager/0.log"
Jul 17 18:59:16.375904 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.375870    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2/alertmanager/0.log"
Jul 17 18:59:16.406951 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.406926    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2/config-reloader/0.log"
Jul 17 18:59:16.433604 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.433576    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2/kube-rbac-proxy-web/0.log"
Jul 17 18:59:16.473615 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.473588    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2/kube-rbac-proxy/0.log"
Jul 17 18:59:16.506653 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.506609    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2/kube-rbac-proxy-metric/0.log"
Jul 17 18:59:16.543291 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.543263    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2/prom-label-proxy/0.log"
Jul 17 18:59:16.588760 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.588731    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dc5f1227-6a6f-4afc-969b-4ea9a04ae5c2/init-config-reloader/0.log"
Jul 17 18:59:16.721192 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.721113    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gpb99_d4f4292b-cc90-4769-bfe6-c49b3b7917c0/kube-state-metrics/0.log"
Jul 17 18:59:16.776919 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.776888    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gpb99_d4f4292b-cc90-4769-bfe6-c49b3b7917c0/kube-rbac-proxy-main/0.log"
Jul 17 18:59:16.806343 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.806317    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-gpb99_d4f4292b-cc90-4769-bfe6-c49b3b7917c0/kube-rbac-proxy-self/0.log"
Jul 17 18:59:16.889226 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:16.889198    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-z9w7t_b02a87df-bb7d-4ca4-b7e2-0babbac79342/monitoring-plugin/0.log"
Jul 17 18:59:17.161016 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.160987    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ldk6v_41157244-5641-4575-a240-93418d028431/node-exporter/0.log"
Jul 17 18:59:17.223488 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.223459    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ldk6v_41157244-5641-4575-a240-93418d028431/kube-rbac-proxy/0.log"
Jul 17 18:59:17.263186 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.263163    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ldk6v_41157244-5641-4575-a240-93418d028431/init-textfile/0.log"
Jul 17 18:59:17.498266 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.498228    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gw5f5_27b36904-0e33-4a42-8ea4-9b8ae5d58955/kube-rbac-proxy-main/0.log"
Jul 17 18:59:17.532627 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.532602    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gw5f5_27b36904-0e33-4a42-8ea4-9b8ae5d58955/kube-rbac-proxy-self/0.log"
Jul 17 18:59:17.582281 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.582245    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"]
Jul 17 18:59:17.582702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.582688    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b26523-1872-4b8c-a5c3-7573ce65cb55" containerName="test-curl-7278-139673085021472"
Jul 17 18:59:17.582702 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.582704    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b26523-1872-4b8c-a5c3-7573ce65cb55" containerName="test-curl-7278-139673085021472"
Jul 17 18:59:17.582814 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.582804    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b26523-1872-4b8c-a5c3-7573ce65cb55" containerName="test-curl-7278-139673085021472"
Jul 17 18:59:17.586233 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.586210    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.592134 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.592112    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lz7w4\"/\"kube-root-ca.crt\""
Jul 17 18:59:17.592373 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.592360    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lz7w4\"/\"default-dockercfg-ft9qh\""
Jul 17 18:59:17.593487 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.593469    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lz7w4\"/\"openshift-service-ca.crt\""
Jul 17 18:59:17.607373 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.607322    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-gw5f5_27b36904-0e33-4a42-8ea4-9b8ae5d58955/openshift-state-metrics/0.log"
Jul 17 18:59:17.607727 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.607682    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"]
Jul 17 18:59:17.651775 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.651699    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-podres\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.651962 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.651802    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzf8x\" (UniqueName: \"kubernetes.io/projected/6c2076b2-4501-4179-8607-31118b4f2f1e-kube-api-access-mzf8x\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.651962 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.651862    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-proc\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.651962 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.651895    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-lib-modules\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.652078 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.652003    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-sys\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752671 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752555    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-sys\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752671 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752624    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-podres\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752675    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzf8x\" (UniqueName: \"kubernetes.io/projected/6c2076b2-4501-4179-8607-31118b4f2f1e-kube-api-access-mzf8x\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752696    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-proc\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752696    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-sys\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752712    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-lib-modules\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752777    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-proc\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752787    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-podres\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.752875 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.752820    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c2076b2-4501-4179-8607-31118b4f2f1e-lib-modules\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.786828 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.786799    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzf8x\" (UniqueName: \"kubernetes.io/projected/6c2076b2-4501-4179-8607-31118b4f2f1e-kube-api-access-mzf8x\") pod \"perf-node-gather-daemonset-gv8sv\" (UID: \"6c2076b2-4501-4179-8607-31118b4f2f1e\") " pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:17.896434 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:17.896390    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:18.065088 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.065058    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"]
Jul 17 18:59:18.066632 ip-10-0-140-23 kubenswrapper[2555]: W0717 18:59:18.066604    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6c2076b2_4501_4179_8607_31118b4f2f1e.slice/crio-32e8235cca452d14f0b397bcfe5d60290ad98225e16611b188ef5f7f84de6d9f WatchSource:0}: Error finding container 32e8235cca452d14f0b397bcfe5d60290ad98225e16611b188ef5f7f84de6d9f: Status 404 returned error can't find the container with id 32e8235cca452d14f0b397bcfe5d60290ad98225e16611b188ef5f7f84de6d9f
Jul 17 18:59:18.163302 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.163265    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv" event={"ID":"6c2076b2-4501-4179-8607-31118b4f2f1e","Type":"ContainerStarted","Data":"3bd05dc4d9c5cf9b8e9e4d7b4bc1749122e969e575a139c0c7f095584f961dd1"}
Jul 17 18:59:18.163486 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.163324    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv" event={"ID":"6c2076b2-4501-4179-8607-31118b4f2f1e","Type":"ContainerStarted","Data":"32e8235cca452d14f0b397bcfe5d60290ad98225e16611b188ef5f7f84de6d9f"}
Jul 17 18:59:18.163486 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.163438    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:18.211764 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.211709    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv" podStartSLOduration=1.2116926000000001 podStartE2EDuration="1.2116926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:59:18.21140205 +0000 UTC m=+3755.796878327" watchObservedRunningTime="2026-07-17 18:59:18.2116926 +0000 UTC m=+3755.797168851"
Jul 17 18:59:18.471266 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.471171    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-stbxg_8c42270d-80c5-4d97-b51a-7dacb92b87c0/prometheus-operator/0.log"
Jul 17 18:59:18.511709 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.511679    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-stbxg_8c42270d-80c5-4d97-b51a-7dacb92b87c0/kube-rbac-proxy/0.log"
Jul 17 18:59:18.774352 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.774322    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cfb69d5b5-cfl9f_8dd1f855-6db3-4d9f-935e-9dfa0e925e65/thanos-query/0.log"
Jul 17 18:59:18.827038 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.827004    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cfb69d5b5-cfl9f_8dd1f855-6db3-4d9f-935e-9dfa0e925e65/kube-rbac-proxy-web/0.log"
Jul 17 18:59:18.868001 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.867970    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cfb69d5b5-cfl9f_8dd1f855-6db3-4d9f-935e-9dfa0e925e65/kube-rbac-proxy/0.log"
Jul 17 18:59:18.923701 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.923676    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cfb69d5b5-cfl9f_8dd1f855-6db3-4d9f-935e-9dfa0e925e65/prom-label-proxy/0.log"
Jul 17 18:59:18.960037 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:18.960005    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cfb69d5b5-cfl9f_8dd1f855-6db3-4d9f-935e-9dfa0e925e65/kube-rbac-proxy-rules/0.log"
Jul 17 18:59:19.004257 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:19.004228    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cfb69d5b5-cfl9f_8dd1f855-6db3-4d9f-935e-9dfa0e925e65/kube-rbac-proxy-metrics/0.log"
Jul 17 18:59:21.398703 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:21.398625    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b47b969d6-rtq7j_c61aee4b-3b03-4849-b944-a39ee9c9163b/console/0.log"
Jul 17 18:59:23.385741 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:23.385707    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nmplq_26ac9501-351d-48de-8572-b583ac2d9a70/dns/0.log"
Jul 17 18:59:23.430139 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:23.430108    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nmplq_26ac9501-351d-48de-8572-b583ac2d9a70/kube-rbac-proxy/0.log"
Jul 17 18:59:23.653146 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:23.653065    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t6frw_44502548-11a1-4a3e-8a4b-07a4f1d5cbb3/dns-node-resolver/0.log"
Jul 17 18:59:24.179555 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:24.179518    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lz7w4/perf-node-gather-daemonset-gv8sv"
Jul 17 18:59:24.306283 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:24.306239    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d4bph_f5736d32-71da-4d48-8c66-fdfd0301ea1f/node-ca/0.log"
Jul 17 18:59:25.719918 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:25.719879    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bfff449b-qmvlc_aa1a105c-005e-47b6-8093-036767bd1b35/kube-auth-proxy/0.log"
Jul 17 18:59:25.872666 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:25.872628    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zkwv7_3ace2039-18db-4d7b-90b0-303fbf522b6c/payload-pre-processing/0.log"
Jul 17 18:59:26.966169 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:26.966072    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4hlgg_725734b7-d548-4d5f-b25f-9fa3e81bddde/serve-healthcheck-canary/0.log"
Jul 17 18:59:28.063408 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:28.063373    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r26rr_b1df721d-260e-4fa4-806b-746e343409f0/kube-rbac-proxy/0.log"
Jul 17 18:59:28.097468 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:28.097439    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r26rr_b1df721d-260e-4fa4-806b-746e343409f0/exporter/0.log"
Jul 17 18:59:28.128455 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:28.128374    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r26rr_b1df721d-260e-4fa4-806b-746e343409f0/extractor/0.log"
Jul 17 18:59:28.826311 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:28.826285    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bfff449b-qmvlc_aa1a105c-005e-47b6-8093-036767bd1b35/kube-auth-proxy/0.log"
Jul 17 18:59:28.947934 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:28.947902    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zkwv7_3ace2039-18db-4d7b-90b0-303fbf522b6c/payload-pre-processing/0.log"
Jul 17 18:59:30.313289 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:30.313252    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84bfff449b-qmvlc_aa1a105c-005e-47b6-8093-036767bd1b35/kube-auth-proxy/0.log"
Jul 17 18:59:30.439750 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:30.439711    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zkwv7_3ace2039-18db-4d7b-90b0-303fbf522b6c/payload-pre-processing/0.log"
Jul 17 18:59:30.529520 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:30.529488    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-fb5cz_1e7f6181-4f4b-4ef7-a269-b847f43ba563/manager/0.log"
Jul 17 18:59:30.801716 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:30.801661    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cpxwg_2cb9e2ac-f565-44c3-a832-4a81a349cc4c/manager/0.log"
Jul 17 18:59:30.981404 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:30.981375    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qwmw7_02eb8a4d-4c18-4663-88ec-a19d0fab22a7/server/0.log"
Jul 17 18:59:31.098877 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:31.098789    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dr8fl_1cfe8d05-ca1c-4a72-9f1b-59277a5a1ed4/manager/0.log"
Jul 17 18:59:31.322860 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:31.322827    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d8bdcf84-72fkb_047e5ea0-35dd-4779-b821-27e1234abbcc/manager/0.log"
Jul 17 18:59:31.346743 ip-10-0-140-23 kubenswrapper[2555]: I0717 18:59:31.346716    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d8bdcf84-72fkb_047e5ea0-35dd-4779-b821-27e1234abbcc/copy-manifests/0.log"