Jul 29 15:11:01.606485 ip-10-0-136-199 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 15:11:01.934057 ip-10-0-136-199 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:11:01.934057 ip-10-0-136-199 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 15:11:01.934057 ip-10-0-136-199 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:11:01.934057 ip-10-0-136-199 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 15:11:01.934057 ip-10-0-136-199 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:11:01.934970 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.934847    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 15:11:01.938931 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938915    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:01.938931 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938932    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938936    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938940    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938943    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938946    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938949    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938951    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938954    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938957    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938964    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938967    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938970    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938973    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938975    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938978    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938981    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938984    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938986    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938990    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.938997    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:01.938998 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939001    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939003    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939007    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939010    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939013    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939015    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939018    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939020    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939030    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939033    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939036    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939039    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939042    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939045    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939047    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939050    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939052    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939055    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939058    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939060    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:01.939505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939063    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939066    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939070    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939074    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939077    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939080    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939083    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939085    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939088    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939091    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939094    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939096    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939099    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939103    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939107    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939109    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939112    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939114    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939117    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:01.940024 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939120    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939122    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939125    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939127    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939132    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939135    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939138    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939142    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939146    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939149    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939152    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939155    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939157    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939160    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939163    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939165    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939168    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939170    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939173    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:01.940505 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939176    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939178    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939193    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939196    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939198    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939201    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939204    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939207    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939210    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939212    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939215    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939218    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939221    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939224    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939226    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939229    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939231    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939234    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939236    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939239    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:01.940963 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939968    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939975    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939978    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939981    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939985    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939988    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939991    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939994    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939997    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.939999    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940002    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940005    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940007    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940010    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940013    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940015    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940018    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940021    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940023    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940026    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:01.941446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940029    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940032    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940036    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940039    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940042    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940045    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940048    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940050    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940053    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940055    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940058    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940060    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940063    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940066    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940070    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940075    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940078    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940081    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940084    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940087    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:01.941943 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940090    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940093    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940095    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940098    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940101    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940104    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940107    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940110    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940112    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940115    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940118    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940120    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940123    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940125    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940128    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940131    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940133    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940136    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940140    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940143    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:01.942446 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940146    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940148    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940151    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940153    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940156    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940158    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940161    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940164    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940166    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940168    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940171    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940174    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940176    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940194    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940197    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940199    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940204    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940207    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940210    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940212    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:01.942929 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940215    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940217    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940220    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940222    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940225    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940228    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940231    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940233    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940236    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940239    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940243    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940247    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940250    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940253    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940255    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940258    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940261    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940263    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940266    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:01.943490 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940634    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940638    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940640    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940643    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940646    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940649    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940652    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940654    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940657    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940660    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940662    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940665    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940668    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940670    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940673    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940675    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940678    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940683    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940685    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940688    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:01.943982 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940691    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940694    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940697    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940699    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940702    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940705    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940707    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940710    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940713    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940715    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940718    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940720    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940723    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940725    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940729    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940731    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940734    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940736    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940739    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940742    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:01.944485 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940744    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940747    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940749    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940752    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940754    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940757    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940759    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940762    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940764    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940769    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940771    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940774    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940777    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940780    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940783    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940786    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940789    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940791    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940794    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940796    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:01.944969 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940799    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940801    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940804    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940806    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940809    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940811    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940815    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940819    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940822    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940824    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940828    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940831    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940834    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940836    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940839    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940841    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940844    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940846    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940849    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:01.945503 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940851    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940854    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940859    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940861    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940864    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940866    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940870    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940873    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940876    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940878    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940881    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940884    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940887    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940890    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940893    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940896    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940899    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940901    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940904    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:01.945955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.940906    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.940976    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.940983    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.940992    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.940999    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941005    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941011    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941017    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941022    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941025    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941028    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941032    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941035    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941038    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941041    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941046    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941049    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941052    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941055    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941061    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941064    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941067    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941070    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941074    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 15:11:01.946427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941078    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941081    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941084    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941087    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941090    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941093    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941096    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941099    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941102    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941106    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941109    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941112    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941115    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941119    2560 flags.go:64] FLAG: --enable-server="true"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941121    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941127    2560 flags.go:64] FLAG: --event-burst="100"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941130    2560 flags.go:64] FLAG: --event-qps="50"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941133    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941136    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941138    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941142    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941145    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941148    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941152    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941156    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 15:11:01.947022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941159    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941162    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941166    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941169    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941172    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941175    2560 flags.go:64] FLAG: --feature-gates=""
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941191    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941195    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941198    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941201    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941205    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941208    2560 flags.go:64] FLAG: --help="false"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941210    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-136-199.ec2.internal"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941214    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941217    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941220    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941223    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941227    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941230    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941232    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941235    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941238    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941241    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941245    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 15:11:01.947668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941248    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941251    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941253    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941256    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941259    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941262    2560 flags.go:64] FLAG: --lock-file=""
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941266    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941269    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941272    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941277    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941281    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941284    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941287    2560 flags.go:64] FLAG: --logging-format="text"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941290    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941293    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941296    2560 flags.go:64] FLAG: --manifest-url=""
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941299    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941304    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941307    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941311    2560 flags.go:64] FLAG: --max-pods="110"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941314    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941317    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941320    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941322    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941325    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 15:11:01.948565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941328    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941331    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941338    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941341    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941344    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941347    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941350    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941357    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941360    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941363    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941366    2560 flags.go:64] FLAG: --port="10250"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941369    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941372    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-015b90ae1791bd5a0"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941375    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941379    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941382    2560 flags.go:64] FLAG: --register-node="true"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941385    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941390    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941393    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941396    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941399    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941403    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941406    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941409    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941412    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 15:11:01.949487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941415    2560 flags.go:64] FLAG: --runonce="false"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941418    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941421    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941424    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941427    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941430    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941433    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941436    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941439    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941442    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941445    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941448    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941451    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941454    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941457    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941461    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941466    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941469    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941472    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941476    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941478    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941482    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941485    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941488    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941492    2560 flags.go:64] FLAG: --v="2"
Jul 29 15:11:01.950097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941496    2560 flags.go:64] FLAG: --version="false"
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941499    2560 flags.go:64] FLAG: --vmodule=""
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941503    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.941506    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941624    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941627    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941630    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941633    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941636    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941639    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941642    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941645    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941649    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941653    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941656    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941659    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941662    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941665    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941667    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:01.950719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941670    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941672    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941674    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941677    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941680    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941682    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941685    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941688    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941690    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941693    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941696    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941698    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941702    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941705    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941708    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941710    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941713    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941716    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941718    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941720    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:01.951257 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941723    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941726    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941729    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941731    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941734    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941738    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941741    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941744    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941747    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941750    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941752    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941755    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941758    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941761    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941763    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941766    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941768    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941771    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941774    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941776    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:01.951750 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941779    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941781    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941784    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941786    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941791    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941793    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941796    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941798    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941801    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941803    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941806    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941809    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941811    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941813    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941816    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941818    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941821    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941823    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941826    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941828    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:01.952295 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941831    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941833    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941836    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941838    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941841    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941843    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941846    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941848    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941851    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941853    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941856    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941858    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941861    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941864    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941866    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941868    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941873    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941875    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941878    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941880    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:01.952787 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941883    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941885    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941887    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.941890    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.942417    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.950753    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.950771    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.950786    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.950792    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950865    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950870    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950874    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950877    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950880    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950883    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:01.953286 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950887    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950889    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950892    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950895    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950898    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950901    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950903    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950906    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950909    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950912    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950914    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950917    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950919    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950922    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950924    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950927    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950930    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950933    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950935    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950938    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:01.953659 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950945    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950948    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950951    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950955    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950957    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950960    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950962    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950965    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950968    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950970    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950973    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950976    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950980    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950984    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950987    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950989    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950992    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950994    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.950997    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951000    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:01.954140 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951002    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951005    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951007    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951010    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951013    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951015    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951019    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951021    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951031    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951035    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951038    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951040    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951044    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951046    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951049    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951053    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951056    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951058    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951061    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951064    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:01.954670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951067    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951069    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951072    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951074    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951077    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951080    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951082    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951085    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951087    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951090    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951093    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951096    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951098    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951101    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951104    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951106    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951109    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951113    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951116    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:01.955170 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951119    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951122    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951124    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951127    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951131    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951135    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951138    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951141    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951145    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951148    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951151    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951154    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951157    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951159    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.951165    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:11:01.955648 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951318    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951324    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951327    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951330    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951333    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951335    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951338    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951341    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951344    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951346    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951349    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951352    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951355    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951357    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951360    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951363    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951365    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951368    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951370    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951373    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:01.956018 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951375    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951378    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951380    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951383    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951386    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951389    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951391    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951394    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951397    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951399    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951402    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951404    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951407    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951410    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951414    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951417    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951420    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951422    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951425    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:01.956676 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951427    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951430    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951433    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951435    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951438    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951440    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951444    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951446    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951449    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951451    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951454    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951456    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951459    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951461    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951464    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951466    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951469    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951472    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951475    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:01.957133 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951479    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951482    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951485    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951487    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951490    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951493    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951496    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951498    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951501    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951503    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951506    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951508    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951511    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951513    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951516    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951519    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951521    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951524    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951526    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951529    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:01.957605 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951532    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951534    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951537    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951539    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951542    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951544    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951547    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951549    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951552    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951554    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951556    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951559    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951561    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951564    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951567    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951569    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951572    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951576    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951579    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:01.958087 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951582    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:01.958544 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:01.951585    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:01.958544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.951589    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:11:01.958544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.951708    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 15:11:01.958544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.956025    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 15:11:01.958544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.957226    2560 server.go:1019] "Starting client certificate rotation"
Jul 29 15:11:01.958544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.957322    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 15:11:01.958544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.957358    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 15:11:01.959860 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.959847    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 15:11:01.962512 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.962490    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 15:11:01.972400 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.972379    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 29 15:11:01.975709 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.975683    2560 log.go:25] "Validated CRI v1 image API"
Jul 29 15:11:01.976595 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.976575    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 15:11:01.979778 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.979756    2560 fs.go:135] Filesystem UUIDs: map[11cb8644-2607-45f4-94d9-5e8e40f384ec:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9944b84f-e192-4ae0-baec-f663913338c4:/dev/nvme0n1p3]
Jul 29 15:11:01.979843 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.979778    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 15:11:01.985132 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.985016    2560 manager.go:216] Machine: {Timestamp:2026-07-29 15:11:01.984452726 +0000 UTC m=+0.278389545 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099194 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e0a2038969e3466e937e691115ace SystemUUID:ec2e0a20-3896-9e34-66e9-37e691115ace BootID:e40e1f42-388c-4435-9e43-cd2bdcc60a62 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ab:f4:20:bb:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ab:f4:20:bb:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:09:54:29:41:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 15:11:01.985132 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.985121    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 15:11:01.985283 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.985221    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 15:11:01.986140 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.986112    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 15:11:01.986291 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.986141    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-199.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 15:11:01.986813 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.986804    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 15:11:01.986843 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.986816    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 15:11:01.986843 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.986824    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 15:11:01.986843 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.986835    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 15:11:01.987666 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.987656    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 15:11:01.990220 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.990209    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 15:11:01.990342 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.990332    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 15:11:01.990874 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.990864    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 15:11:01.990904 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.990880    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 15:11:01.990904 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.990891    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 15:11:01.990904 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.990900    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 29 15:11:01.991013 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.990908    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 15:11:01.991311 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.991297    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 15:11:01.991380 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.991325    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 15:11:01.995344 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.995322    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 15:11:01.995465 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.995448    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 15:11:01.996943 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.996931    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 15:11:01.996997 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.996961    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 15:11:01.998439 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998425    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 15:11:01.998495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998443    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 15:11:01.998495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998450    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 15:11:01.998495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998457    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 15:11:01.998495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998471    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 15:11:01.998495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998479    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 15:11:01.998495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998488    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 15:11:01.998495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998494    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 15:11:01.998732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998502    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 15:11:01.998732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998508    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 15:11:01.998732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998518    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 15:11:01.998732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.998524    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 15:11:01.999137 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.999127    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 15:11:01.999174 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:01.999138    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 15:11:02.002663 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.002646    2560 server.go:1268] "Started kubelet"
Jul 29 15:11:02.002768 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.002737    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 15:11:02.002768 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.002724    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 15:11:02.002876 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.002797    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 15:11:02.003671 ip-10-0-136-199 systemd[1]: Started Kubernetes Kubelet.
Jul 29 15:11:02.004052 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.004032    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-199.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 15:11:02.004326 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.004289    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 15:11:02.004391 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.004334    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-199.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 15:11:02.004865 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.004680    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 15:11:02.005577 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.005559    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 15:11:02.010956 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.010928    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 15:11:02.012198 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.012184    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 15:11:02.012362 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.012282    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 15:11:02.012434 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.012419    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.012858 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.012838    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 15:11:02.012858 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.012859    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 15:11:02.013022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.012838    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 15:11:02.013082 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013043    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 15:11:02.013082 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013053    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 15:11:02.013167 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013130    2560 factory.go:55] Registering systemd factory
Jul 29 15:11:02.013237 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013174    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 29 15:11:02.013467 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013438    2560 factory.go:153] Registering CRI-O factory
Jul 29 15:11:02.013467 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013458    2560 factory.go:223] Registration of the crio container factory successfully
Jul 29 15:11:02.013616 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013510    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 15:11:02.013616 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013535    2560 factory.go:103] Registering Raw factory
Jul 29 15:11:02.013616 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.013554    2560 manager.go:1196] Started watching for new ooms in manager
Jul 29 15:11:02.016556 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.016516    2560 manager.go:349] Starting recovery of all containers
Jul 29 15:11:02.016791 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.013589    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-199.ec2.internal.18c6cb538cb8a9f7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-199.ec2.internal,UID:ip-10-0-136-199.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-199.ec2.internal,},FirstTimestamp:2026-07-29 15:11:02.002616823 +0000 UTC m=+0.296553620,LastTimestamp:2026-07-29 15:11:02.002616823 +0000 UTC m=+0.296553620,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-199.ec2.internal,}"
Jul 29 15:11:02.020225 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.020176    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-199.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 15:11:02.020225 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.020212    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 15:11:02.030294 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.030272    2560 manager.go:354] Recovery completed
Jul 29 15:11:02.034872 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.034859    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:02.037063 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.037037    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:02.037129 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.037079    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:02.037129 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.037092    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:02.037777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.037762    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 15:11:02.037777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.037774    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 15:11:02.037777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.037775    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:02.037919 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.037790    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 15:11:02.038565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.038549    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:02.038636 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.038579    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:02.038636 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.038594    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:02.039392 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.039311    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-199.ec2.internal.18c6cb538ec64727  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-199.ec2.internal,UID:ip-10-0-136-199.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-199.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-199.ec2.internal,},FirstTimestamp:2026-07-29 15:11:02.037063463 +0000 UTC m=+0.331000258,LastTimestamp:2026-07-29 15:11:02.037063463 +0000 UTC m=+0.331000258,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-199.ec2.internal,}"
Jul 29 15:11:02.040665 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.040651    2560 policy_none.go:49] "None policy: Start"
Jul 29 15:11:02.040724 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.040668    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 15:11:02.040724 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.040680    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 15:11:02.042780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.042769    2560 policy_none.go:47] "Start"
Jul 29 15:11:02.051511 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.051295    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-199.ec2.internal.18c6cb538ec69a7c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-199.ec2.internal,UID:ip-10-0-136-199.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-199.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-199.ec2.internal,},FirstTimestamp:2026-07-29 15:11:02.037084796 +0000 UTC m=+0.331021592,LastTimestamp:2026-07-29 15:11:02.037084796 +0000 UTC m=+0.331021592,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-199.ec2.internal,}"
Jul 29 15:11:02.054210 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.054177    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mcnww"
Jul 29 15:11:02.059594 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.059523    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-199.ec2.internal.18c6cb538ec6c7ea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-199.ec2.internal,UID:ip-10-0-136-199.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-199.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-199.ec2.internal,},FirstTimestamp:2026-07-29 15:11:02.037096426 +0000 UTC m=+0.331033222,LastTimestamp:2026-07-29 15:11:02.037096426 +0000 UTC m=+0.331033222,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-199.ec2.internal,}"
Jul 29 15:11:02.063990 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.063968    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mcnww"
Jul 29 15:11:02.069054 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.068983    2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-199.ec2.internal.18c6cb538ec64727\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-199.ec2.internal.18c6cb538ec64727  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-199.ec2.internal,UID:ip-10-0-136-199.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-199.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-199.ec2.internal,},FirstTimestamp:2026-07-29 15:11:02.037063463 +0000 UTC m=+0.331000258,LastTimestamp:2026-07-29 15:11:02.038564544 +0000 UTC m=+0.332501340,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-199.ec2.internal,}"
Jul 29 15:11:02.082877 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.082854    2560 manager.go:337] "Starting Device Plugin manager"
Jul 29 15:11:02.082980 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.082903    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 15:11:02.082980 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.082918    2560 server.go:87] "Starting device plugin registration server"
Jul 29 15:11:02.083204 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.083191    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 15:11:02.083285 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.083208    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 15:11:02.083353 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.083313    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 15:11:02.083897 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.083829    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 15:11:02.083897 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.083854    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 15:11:02.084058 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.084016    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 15:11:02.084107 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.084094    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.158250 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.158203    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 15:11:02.168451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.159616    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 15:11:02.168451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.159665    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 15:11:02.168451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.159698    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 15:11:02.168451 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.159743    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 15:11:02.168451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.162957    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:02.183544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.183523    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:02.184659 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.184602    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:02.184659 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.184635    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:02.184659 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.184646    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:02.184812 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.184670    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.195512 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.195489    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.195568 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.195518    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-199.ec2.internal\": node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.259567 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.259519    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.260622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.260601    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal"]
Jul 29 15:11:02.260715 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.260682    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:02.262591 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.262575    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:02.262671 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.262606    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:02.262671 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.262616    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:02.265231 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.265217    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:02.265362 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.265347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.265411 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.265380    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:02.266030 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.266014    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:02.266145 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.266027    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:02.266145 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.266046    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:02.266145 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.266054    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:02.266145 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.266063    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:02.266145 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.266066    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:02.268610 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.268593    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.268666 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.268625    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:02.269782 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.269762    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:02.269887 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.269799    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:02.269887 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.269817    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:02.293338 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.293312    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-199.ec2.internal\" not found" node="ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.297957 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.297941    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-199.ec2.internal\" not found" node="ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.314382 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.314360    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/139e51abe32fc6d140a5f3ceeea4020f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal\" (UID: \"139e51abe32fc6d140a5f3ceeea4020f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.314452 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.314386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/139e51abe32fc6d140a5f3ceeea4020f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal\" (UID: \"139e51abe32fc6d140a5f3ceeea4020f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.314452 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.314405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf62266a6ae3b77620eda580b89764b9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-199.ec2.internal\" (UID: \"cf62266a6ae3b77620eda580b89764b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.359866 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.359838    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.414790 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.414754    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/139e51abe32fc6d140a5f3ceeea4020f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal\" (UID: \"139e51abe32fc6d140a5f3ceeea4020f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.414790 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.414788    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/139e51abe32fc6d140a5f3ceeea4020f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal\" (UID: \"139e51abe32fc6d140a5f3ceeea4020f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.414993 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.414807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf62266a6ae3b77620eda580b89764b9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-199.ec2.internal\" (UID: \"cf62266a6ae3b77620eda580b89764b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.414993 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.414837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf62266a6ae3b77620eda580b89764b9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-199.ec2.internal\" (UID: \"cf62266a6ae3b77620eda580b89764b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.414993 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.414875    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/139e51abe32fc6d140a5f3ceeea4020f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal\" (UID: \"139e51abe32fc6d140a5f3ceeea4020f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.414993 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.414882    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/139e51abe32fc6d140a5f3ceeea4020f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal\" (UID: \"139e51abe32fc6d140a5f3ceeea4020f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.460714 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.460639    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.563869 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.563829    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.598290 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.598247    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.600801 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.600782    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.665903 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.665863    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.769486 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:02.769402    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-199.ec2.internal\" not found"
Jul 29 15:11:02.817975 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.817948    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:02.882625 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.882599    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:02.913579 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.913547    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.933197 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.933169    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 15:11:02.934691 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.934670    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal"
Jul 29 15:11:02.946598 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.946572    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 15:11:02.956394 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.956369    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 15:11:02.956583 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.956529    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 15:11:02.956583 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.956529    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 15:11:02.991579 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.991531    2560 apiserver.go:52] "Watching apiserver"
Jul 29 15:11:02.999739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:02.999715    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 15:11:03.000086 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.000065    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv","openshift-dns/node-resolver-6z2cx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal","openshift-multus/multus-dpmmq","openshift-multus/network-metrics-daemon-gnjvk","kube-system/global-pull-secret-syncer-vvz9k","kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal","openshift-cluster-node-tuning-operator/tuned-f2kcq","openshift-image-registry/node-ca-6b67r","openshift-multus/multus-additional-cni-plugins-tlxfp","openshift-network-diagnostics/network-check-target-b7zwb","openshift-network-operator/iptables-alerter-64l8z","openshift-ovn-kubernetes/ovnkube-node-sscmw","kube-system/konnectivity-agent-vd2hl"]
Jul 29 15:11:03.002875 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.002857    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.005249 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.005229    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.007605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.007586    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.009351 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.009326    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 15:11:03.009467 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.009446    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 15:11:03.009558 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.009502    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 15:11:03.009558 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.009523    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 15:11:03.009558 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.009527    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 15:11:03.009856 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.009838    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 15:11:03.009978 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.009960    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.010188 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010171    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 15:11:03.010331 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010318    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8vgg9\""
Jul 29 15:11:03.010533 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010520    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 15:11:03.010966 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010947    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 15:11:03.011046 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010964    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n9hld\""
Jul 29 15:11:03.011046 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010967    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 15:11:03.011046 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010999    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 15:11:03.011046 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.011022    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 15:11:03.011046 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.011044    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 15:11:03.011259 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.010999    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nb92h\""
Jul 29 15:11:03.012364 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.012339    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:03.012453 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.012397    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:03.012514 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.012458    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 15:11:03.012615 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.012598    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 15:11:03.013351 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.013329    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vt2bp\""
Jul 29 15:11:03.014922 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.014899    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.017242 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017212    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-kubernetes\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.017361 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017296    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.017429 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-cni-netd\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.017429 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017416    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbvch\" (UniqueName: \"kubernetes.io/projected/c6e6ae73-f87c-4666-9d7b-28584bb70925-kube-api-access-pbvch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.017531 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017449    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-cnibin\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.017531 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017485    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-netns\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.017531 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017519    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-kubelet\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.017714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017553    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-etc-kubernetes\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.017714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017598    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.017714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b62sh\" (UniqueName: \"kubernetes.io/projected/85c24726-0eb1-4ccf-83d9-9b4643142d10-kube-api-access-b62sh\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.017714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017702    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-cni-bin\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.017927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017738    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-log-socket\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.017927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017773    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-var-lib-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.017927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017778    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:11:03.017927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017806    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-system-cni-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.017927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017838    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-multus-certs\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.017927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017870    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cni-binary-copy\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.017927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017904    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkltj\" (UniqueName: \"kubernetes.io/projected/eebdf1b2-5e28-4f3e-a36d-7687e491314d-kube-api-access-wkltj\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017939    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-env-overrides\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017972    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-slash\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.017990    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018009    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-daemon-config\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018054    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018117    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-55v2m\""
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-systemd\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.018303 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018288    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-lib-modules\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.018647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018330    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-cni-bin\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.018647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.018647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-kubelet\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.018647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-socket-dir-parent\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.018647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018563    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgfhs\" (UniqueName: \"kubernetes.io/projected/2415e25c-6267-4eb0-bb24-11872c713139-kube-api-access-vgfhs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:03.018647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018596    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-cni-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.018647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018639    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysctl-conf\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.018923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018678    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-var-lib-kubelet\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.018923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-tmp\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.018923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-ovn\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.018923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018827    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-run-netns\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.018923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-systemd\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.018923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018911    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-hostroot\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-host\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.018990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-tuned\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-system-cni-dir\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-os-release\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-os-release\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019170    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysctl-d\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.019212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019187    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.019553 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019232    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovn-node-metrics-cert\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019617    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-k8s-cni-cncf-io\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019652    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-modprobe-d\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-run-ovn-kubernetes\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovnkube-script-lib\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-conf-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019741    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-systemd-units\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-node-log\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdmsq\" (UniqueName: \"kubernetes.io/projected/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-kube-api-access-bdmsq\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019806    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019829    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-etc-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/358e0c8d-7a8b-4876-9b32-2c379aeccecb-agent-certs\") pod \"konnectivity-agent-vd2hl\" (UID: \"358e0c8d-7a8b-4876-9b32-2c379aeccecb\") " pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019930    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/358e0c8d-7a8b-4876-9b32-2c379aeccecb-konnectivity-ca\") pod \"konnectivity-agent-vd2hl\" (UID: \"358e0c8d-7a8b-4876-9b32-2c379aeccecb\") " pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85c24726-0eb1-4ccf-83d9-9b4643142d10-cni-binary-copy\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.019997    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-cni-multus\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-sys\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020094    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 15:11:03.020108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020103    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysconfig\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.020800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-run\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.020800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020177    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cnibin\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.020800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovnkube-config\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.020800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020320    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 15:11:03.020800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.020570    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.021100 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.021086    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 15:11:03.021139 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.021119    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b5282\""
Jul 29 15:11:03.023228 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.023211    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.023540 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.023517    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-psjgp\""
Jul 29 15:11:03.023642 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.023619    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:11:03.023701 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.023679    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 15:11:03.023815 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.023790    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 15:11:03.025390 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.025377    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kg7nz\""
Jul 29 15:11:03.025738 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.025726    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.025940 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.025922    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 15:11:03.026911 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.026895    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 15:11:03.028147 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.028131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:03.028234 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.028213    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:03.028519 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.028497    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 15:11:03.028622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.028518    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 15:11:03.028622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.028569    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q5vnt\""
Jul 29 15:11:03.028622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.028615    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 15:11:03.030636 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.030617    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.030701 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.030671    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:03.030876 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.030863    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 15:11:03.055091 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.055075    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lzvpx"
Jul 29 15:11:03.065310 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.065284    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lzvpx"
Jul 29 15:11:03.068796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.068759    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 15:06:02 +0000 UTC" deadline="2028-03-06 02:34:13.519422537 +0000 UTC"
Jul 29 15:11:03.068796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.068794    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14051h23m10.450630966s"
Jul 29 15:11:03.113957 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.113933    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 15:11:03.121720 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.121692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgfhs\" (UniqueName: \"kubernetes.io/projected/2415e25c-6267-4eb0-bb24-11872c713139-kube-api-access-vgfhs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:03.121873 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.121735    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-cni-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.121943 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.121900    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-cni-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.122027 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122011    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysctl-conf\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122077 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-var-lib-kubelet\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122077 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-tmp\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122167 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122087    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-ovn\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.122167 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122109    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-run-netns\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.122167 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122130    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-systemd\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.122167 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-hostroot\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122162    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-var-lib-kubelet\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122239    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-hostroot\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122239    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysctl-conf\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122245    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-run-netns\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122219    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-systemd\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122179    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-host\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122176    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-ovn\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-host\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-tuned\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122337    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-system-cni-dir\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-os-release\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122395    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52224984-89d4-44bb-8119-2a95db6c1d03-host\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.122409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122404    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-system-cni-dir\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52224984-89d4-44bb-8119-2a95db6c1d03-serviceca\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gcfx\" (UniqueName: \"kubernetes.io/projected/49b10c1c-0978-4e3c-9145-d40da94ad15f-kube-api-access-8gcfx\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122452    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-os-release\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122477    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-os-release\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122513    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122528    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysctl-d\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122556    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-os-release\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.122612    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.122713    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:03.622670198 +0000 UTC m=+1.916607017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122714    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysctl-d\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovn-node-metrics-cert\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122866    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-kubelet-config\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122901    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-k8s-cni-cncf-io\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-modprobe-d\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.123221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-run-ovn-kubernetes\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.122989    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovnkube-script-lib\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-conf-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-systemd-units\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123070    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-node-log\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-etc-selinux\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123211    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdmsq\" (UniqueName: \"kubernetes.io/projected/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-kube-api-access-bdmsq\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123241    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-etc-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123290    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/358e0c8d-7a8b-4876-9b32-2c379aeccecb-agent-certs\") pod \"konnectivity-agent-vd2hl\" (UID: \"358e0c8d-7a8b-4876-9b32-2c379aeccecb\") " pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/358e0c8d-7a8b-4876-9b32-2c379aeccecb-konnectivity-ca\") pod \"konnectivity-agent-vd2hl\" (UID: \"358e0c8d-7a8b-4876-9b32-2c379aeccecb\") " pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123395    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-systemd-units\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123433    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85c24726-0eb1-4ccf-83d9-9b4643142d10-cni-binary-copy\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-cni-multus\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123490    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-sys\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.124035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123514    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysconfig\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-run\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123561    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cnibin\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovnkube-config\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-node-log\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35b71621-959c-4601-9800-f5e3f26182fa-tmp-dir\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123696    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-socket-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123721    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123747    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-kubernetes\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123772    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-cni-netd\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123826    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbvch\" (UniqueName: \"kubernetes.io/projected/c6e6ae73-f87c-4666-9d7b-28584bb70925-kube-api-access-pbvch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-registration-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123879    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-cnibin\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-netns\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-kubelet\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123964    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-etc-kubernetes\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.124816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.123994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124021    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llpgv\" (UniqueName: \"kubernetes.io/projected/35b71621-959c-4601-9800-f5e3f26182fa-kube-api-access-llpgv\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124050    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-sys-fs\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62sh\" (UniqueName: \"kubernetes.io/projected/85c24726-0eb1-4ccf-83d9-9b4643142d10-kube-api-access-b62sh\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124090    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovnkube-script-lib\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124109    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-cni-bin\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124140    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124149    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovnkube-config\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124168    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-log-socket\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-k8s-cni-cncf-io\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124208    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-conf-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124229    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-log-socket\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124241    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/35b71621-959c-4601-9800-f5e3f26182fa-hosts-file\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124291    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-etc-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-var-lib-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-kubernetes\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124346    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-run-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.125501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124546    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/358e0c8d-7a8b-4876-9b32-2c379aeccecb-konnectivity-ca\") pod \"konnectivity-agent-vd2hl\" (UID: \"358e0c8d-7a8b-4876-9b32-2c379aeccecb\") " pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124800    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85c24726-0eb1-4ccf-83d9-9b4643142d10-cni-binary-copy\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-dbus\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-system-cni-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-multus-certs\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124879    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124888    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cni-binary-copy\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkltj\" (UniqueName: \"kubernetes.io/projected/eebdf1b2-5e28-4f3e-a36d-7687e491314d-kube-api-access-wkltj\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124940    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-cni-netd\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-env-overrides\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49b10c1c-0978-4e3c-9145-d40da94ad15f-host-slash\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-slash\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-daemon-config\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-systemd\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-lib-modules\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125163    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjhgd\" (UniqueName: \"kubernetes.io/projected/52224984-89d4-44bb-8119-2a95db6c1d03-kube-api-access-cjhgd\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvxb7\" (UniqueName: \"kubernetes.io/projected/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-kube-api-access-jvxb7\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.126409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125216    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-netns\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-cni-multus\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125308    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-kubelet\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125340    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-cni-bin\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125362    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-etc-kubernetes\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-run\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125423    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-sysconfig\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125434    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-slash\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.124231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-run-ovn-kubernetes\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125484    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-var-lib-openvswitch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-system-cni-dir\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125598    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125644    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-run-multus-certs\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125664    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-modprobe-d\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-host-var-lib-cni-bin\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125707    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125734    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-kubelet\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-sys\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125925    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cnibin\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125950    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-daemon-config\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.125981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-socket-dir-parent\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49b10c1c-0978-4e3c-9145-d40da94ad15f-iptables-alerter-script\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-systemd\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126050    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-device-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126166    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-kubelet\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126193    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-tmp\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126212    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6e6ae73-f87c-4666-9d7b-28584bb70925-host-cni-bin\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126277    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eebdf1b2-5e28-4f3e-a36d-7687e491314d-cni-binary-copy\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126295    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-cnibin\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-lib-modules\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/85c24726-0eb1-4ccf-83d9-9b4643142d10-multus-socket-dir-parent\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6e6ae73-f87c-4666-9d7b-28584bb70925-ovn-node-metrics-cert\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126473    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eebdf1b2-5e28-4f3e-a36d-7687e491314d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6e6ae73-f87c-4666-9d7b-28584bb70925-env-overrides\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.126502    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-etc-tuned\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.127739 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.127319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/358e0c8d-7a8b-4876-9b32-2c379aeccecb-agent-certs\") pod \"konnectivity-agent-vd2hl\" (UID: \"358e0c8d-7a8b-4876-9b32-2c379aeccecb\") " pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.131786 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.131766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgfhs\" (UniqueName: \"kubernetes.io/projected/2415e25c-6267-4eb0-bb24-11872c713139-kube-api-access-vgfhs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:03.133340 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.133314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdmsq\" (UniqueName: \"kubernetes.io/projected/0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8-kube-api-access-bdmsq\") pod \"tuned-f2kcq\" (UID: \"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8\") " pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.133779 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.133761    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkltj\" (UniqueName: \"kubernetes.io/projected/eebdf1b2-5e28-4f3e-a36d-7687e491314d-kube-api-access-wkltj\") pod \"multus-additional-cni-plugins-tlxfp\" (UID: \"eebdf1b2-5e28-4f3e-a36d-7687e491314d\") " pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.134024 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.134008    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbvch\" (UniqueName: \"kubernetes.io/projected/c6e6ae73-f87c-4666-9d7b-28584bb70925-kube-api-access-pbvch\") pod \"ovnkube-node-sscmw\" (UID: \"c6e6ae73-f87c-4666-9d7b-28584bb70925\") " pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.143633 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.143607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b62sh\" (UniqueName: \"kubernetes.io/projected/85c24726-0eb1-4ccf-83d9-9b4643142d10-kube-api-access-b62sh\") pod \"multus-dpmmq\" (UID: \"85c24726-0eb1-4ccf-83d9-9b4643142d10\") " pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.226497 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjhgd\" (UniqueName: \"kubernetes.io/projected/52224984-89d4-44bb-8119-2a95db6c1d03-kube-api-access-cjhgd\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.226497 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226501    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxb7\" (UniqueName: \"kubernetes.io/projected/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-kube-api-access-jvxb7\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49b10c1c-0978-4e3c-9145-d40da94ad15f-iptables-alerter-script\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226563    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-device-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52224984-89d4-44bb-8119-2a95db6c1d03-host\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226623    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52224984-89d4-44bb-8119-2a95db6c1d03-serviceca\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226646    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gcfx\" (UniqueName: \"kubernetes.io/projected/49b10c1c-0978-4e3c-9145-d40da94ad15f-kube-api-access-8gcfx\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-kubelet-config\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226723    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226722    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52224984-89d4-44bb-8119-2a95db6c1d03-host\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226740    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-device-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.226758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-etc-selinux\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35b71621-959c-4601-9800-f5e3f26182fa-tmp-dir\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-socket-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226847    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-kubelet-config\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-registration-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llpgv\" (UniqueName: \"kubernetes.io/projected/35b71621-959c-4601-9800-f5e3f26182fa-kube-api-access-llpgv\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226979    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-etc-selinux\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-sys-fs\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227030    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.227043    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/35b71621-959c-4601-9800-f5e3f26182fa-hosts-file\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227088    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-sys-fs\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227142    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-dbus\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227089    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-dbus\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-registration-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227337 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.227161    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret podName:025b10db-422e-4bbc-b0b2-e4dfa0c5fecf nodeName:}" failed. No retries permitted until 2026-07-29 15:11:03.727142057 +0000 UTC m=+2.021078846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret") pod "global-pull-secret-syncer-vvz9k" (UID: "025b10db-422e-4bbc-b0b2-e4dfa0c5fecf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:03.227917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227161    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52224984-89d4-44bb-8119-2a95db6c1d03-serviceca\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.227917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35b71621-959c-4601-9800-f5e3f26182fa-tmp-dir\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.227917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.226996    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-socket-dir\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.227917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227218    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49b10c1c-0978-4e3c-9145-d40da94ad15f-iptables-alerter-script\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.227917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227223    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/35b71621-959c-4601-9800-f5e3f26182fa-hosts-file\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.227917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49b10c1c-0978-4e3c-9145-d40da94ad15f-host-slash\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.227917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.227277    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49b10c1c-0978-4e3c-9145-d40da94ad15f-host-slash\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.238861 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.238832    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:03.238861 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.238867    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:03.239079 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.238884    2560 projected.go:196] Error preparing data for projected volume kube-api-access-f7j65 for pod openshift-network-diagnostics/network-check-target-b7zwb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:03.239079 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.238968    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65 podName:43dda4c6-ff99-48e8-a28b-09250dfe09f1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:03.73894559 +0000 UTC m=+2.032882397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f7j65" (UniqueName: "kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65") pod "network-check-target-b7zwb" (UID: "43dda4c6-ff99-48e8-a28b-09250dfe09f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:03.240776 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.240753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gcfx\" (UniqueName: \"kubernetes.io/projected/49b10c1c-0978-4e3c-9145-d40da94ad15f-kube-api-access-8gcfx\") pod \"iptables-alerter-64l8z\" (UID: \"49b10c1c-0978-4e3c-9145-d40da94ad15f\") " pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.240902 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.240880    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llpgv\" (UniqueName: \"kubernetes.io/projected/35b71621-959c-4601-9800-f5e3f26182fa-kube-api-access-llpgv\") pod \"node-resolver-6z2cx\" (UID: \"35b71621-959c-4601-9800-f5e3f26182fa\") " pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.241027 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.241001    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjhgd\" (UniqueName: \"kubernetes.io/projected/52224984-89d4-44bb-8119-2a95db6c1d03-kube-api-access-cjhgd\") pod \"node-ca-6b67r\" (UID: \"52224984-89d4-44bb-8119-2a95db6c1d03\") " pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.242696 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.242671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxb7\" (UniqueName: \"kubernetes.io/projected/7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242-kube-api-access-jvxb7\") pod \"aws-ebs-csi-driver-node-dx2lv\" (UID: \"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.333546 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.333464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tlxfp"
Jul 29 15:11:03.350345 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.350314    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:03.373090 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.373050    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:03.386104 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.386082    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dpmmq"
Jul 29 15:11:03.403304 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.403276    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f2kcq"
Jul 29 15:11:03.410712 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.410682    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:03.419278 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.419253    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6b67r"
Jul 29 15:11:03.427970 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:03.427938    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139e51abe32fc6d140a5f3ceeea4020f.slice/crio-971fe6a8c55afbe5884c44b758f61b214a2c9c6c2c3931195753be60af84bc07 WatchSource:0}: Error finding container 971fe6a8c55afbe5884c44b758f61b214a2c9c6c2c3931195753be60af84bc07: Status 404 returned error can't find the container with id 971fe6a8c55afbe5884c44b758f61b214a2c9c6c2c3931195753be60af84bc07
Jul 29 15:11:03.430106 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:03.430082    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52224984_89d4_44bb_8119_2a95db6c1d03.slice/crio-3dc00f9bf4924897d2f7e7f889f1fd917d4b4becb8069b14e7f21f991a950d68 WatchSource:0}: Error finding container 3dc00f9bf4924897d2f7e7f889f1fd917d4b4becb8069b14e7f21f991a950d68: Status 404 returned error can't find the container with id 3dc00f9bf4924897d2f7e7f889f1fd917d4b4becb8069b14e7f21f991a950d68
Jul 29 15:11:03.433067 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.433051    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:11:03.445290 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.445261    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-64l8z"
Jul 29 15:11:03.455245 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:03.455212    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b10c1c_0978_4e3c_9145_d40da94ad15f.slice/crio-8568406b97ca3f35b7a91880a12c578a0700fbda6e8eaa838364836246913aff WatchSource:0}: Error finding container 8568406b97ca3f35b7a91880a12c578a0700fbda6e8eaa838364836246913aff: Status 404 returned error can't find the container with id 8568406b97ca3f35b7a91880a12c578a0700fbda6e8eaa838364836246913aff
Jul 29 15:11:03.466647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.466626    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6z2cx"
Jul 29 15:11:03.471193 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.470984    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv"
Jul 29 15:11:03.479094 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:03.479063    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b71621_959c_4601_9800_f5e3f26182fa.slice/crio-b2d320637ed9108db30d4d3ac12bdf082e8746521dda35221550f9f587759895 WatchSource:0}: Error finding container b2d320637ed9108db30d4d3ac12bdf082e8746521dda35221550f9f587759895: Status 404 returned error can't find the container with id b2d320637ed9108db30d4d3ac12bdf082e8746521dda35221550f9f587759895
Jul 29 15:11:03.480353 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:03.480326    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1ef0cc_f0c1_43b2_9cfb_93bd131b2242.slice/crio-d6fda883fd58f4a4016a7b4dc4d3f0ae0f364b98181fc4965b1bbcf3e6d88935 WatchSource:0}: Error finding container d6fda883fd58f4a4016a7b4dc4d3f0ae0f364b98181fc4965b1bbcf3e6d88935: Status 404 returned error can't find the container with id d6fda883fd58f4a4016a7b4dc4d3f0ae0f364b98181fc4965b1bbcf3e6d88935
Jul 29 15:11:03.631169 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.631066    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:03.631349 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.631236    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:03.631349 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.631292    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:04.631278225 +0000 UTC m=+2.925215007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:03.732585 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.732540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:03.732756 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.732703    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:03.732811 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.732775    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret podName:025b10db-422e-4bbc-b0b2-e4dfa0c5fecf nodeName:}" failed. No retries permitted until 2026-07-29 15:11:04.732758677 +0000 UTC m=+3.026695464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret") pod "global-pull-secret-syncer-vvz9k" (UID: "025b10db-422e-4bbc-b0b2-e4dfa0c5fecf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:03.740387 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:03.740355    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e6ae73_f87c_4666_9d7b_28584bb70925.slice/crio-634e111ffba57e225df66313afe3ea70ac172031b3b8f34514eca1a877b43c17 WatchSource:0}: Error finding container 634e111ffba57e225df66313afe3ea70ac172031b3b8f34514eca1a877b43c17: Status 404 returned error can't find the container with id 634e111ffba57e225df66313afe3ea70ac172031b3b8f34514eca1a877b43c17
Jul 29 15:11:03.741012 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:03.740986    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbd1463_c4bc_44fb_8f42_c7e6dc518dc8.slice/crio-be74823b8a86827c1eebca183e3a708485d9b3636b0768f268952d5960ec6ef2 WatchSource:0}: Error finding container be74823b8a86827c1eebca183e3a708485d9b3636b0768f268952d5960ec6ef2: Status 404 returned error can't find the container with id be74823b8a86827c1eebca183e3a708485d9b3636b0768f268952d5960ec6ef2
Jul 29 15:11:03.833567 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:03.833525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:03.833754 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.833682    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:03.833754 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.833702    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:03.833754 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.833713    2560 projected.go:196] Error preparing data for projected volume kube-api-access-f7j65 for pod openshift-network-diagnostics/network-check-target-b7zwb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:03.833870 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:03.833771    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65 podName:43dda4c6-ff99-48e8-a28b-09250dfe09f1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:04.833754277 +0000 UTC m=+3.127691080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7j65" (UniqueName: "kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65") pod "network-check-target-b7zwb" (UID: "43dda4c6-ff99-48e8-a28b-09250dfe09f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:04.068652 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.068475    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 15:06:03 +0000 UTC" deadline="2028-01-22 13:40:17.663797395 +0000 UTC"
Jul 29 15:11:04.068652 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.068589    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13006h29m13.595213836s"
Jul 29 15:11:04.098691 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.098649    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:04.123573 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:04.123536    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebdf1b2_5e28_4f3e_a36d_7687e491314d.slice/crio-767c07b6718d9561d82e169c23a0bb574dc1d2ef017aefc81e498107502723a8 WatchSource:0}: Error finding container 767c07b6718d9561d82e169c23a0bb574dc1d2ef017aefc81e498107502723a8: Status 404 returned error can't find the container with id 767c07b6718d9561d82e169c23a0bb574dc1d2ef017aefc81e498107502723a8
Jul 29 15:11:04.167523 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.167454    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"634e111ffba57e225df66313afe3ea70ac172031b3b8f34514eca1a877b43c17"}
Jul 29 15:11:04.167694 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:04.167672    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c24726_0eb1_4ccf_83d9_9b4643142d10.slice/crio-94d20e300a2a98ec34eee0aacba6473ec1cd6bfb98a0b228424064d533b474a0 WatchSource:0}: Error finding container 94d20e300a2a98ec34eee0aacba6473ec1cd6bfb98a0b228424064d533b474a0: Status 404 returned error can't find the container with id 94d20e300a2a98ec34eee0aacba6473ec1cd6bfb98a0b228424064d533b474a0
Jul 29 15:11:04.170600 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.170566    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv" event={"ID":"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242","Type":"ContainerStarted","Data":"d6fda883fd58f4a4016a7b4dc4d3f0ae0f364b98181fc4965b1bbcf3e6d88935"}
Jul 29 15:11:04.172879 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.172852    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6z2cx" event={"ID":"35b71621-959c-4601-9800-f5e3f26182fa","Type":"ContainerStarted","Data":"b2d320637ed9108db30d4d3ac12bdf082e8746521dda35221550f9f587759895"}
Jul 29 15:11:04.178286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.178239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-64l8z" event={"ID":"49b10c1c-0978-4e3c-9145-d40da94ad15f","Type":"ContainerStarted","Data":"8568406b97ca3f35b7a91880a12c578a0700fbda6e8eaa838364836246913aff"}
Jul 29 15:11:04.180925 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.180896    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6b67r" event={"ID":"52224984-89d4-44bb-8119-2a95db6c1d03","Type":"ContainerStarted","Data":"3dc00f9bf4924897d2f7e7f889f1fd917d4b4becb8069b14e7f21f991a950d68"}
Jul 29 15:11:04.183870 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.183843    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal" event={"ID":"139e51abe32fc6d140a5f3ceeea4020f","Type":"ContainerStarted","Data":"971fe6a8c55afbe5884c44b758f61b214a2c9c6c2c3931195753be60af84bc07"}
Jul 29 15:11:04.185486 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.185461    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerStarted","Data":"767c07b6718d9561d82e169c23a0bb574dc1d2ef017aefc81e498107502723a8"}
Jul 29 15:11:04.187104 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.187076    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f2kcq" event={"ID":"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8","Type":"ContainerStarted","Data":"be74823b8a86827c1eebca183e3a708485d9b3636b0768f268952d5960ec6ef2"}
Jul 29 15:11:04.641049 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.640985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:04.641251 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.641179    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:04.641338 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.641294    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:06.641261186 +0000 UTC m=+4.935197992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:04.743012 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.742342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:04.743012 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.742559    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:04.743012 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.742626    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret podName:025b10db-422e-4bbc-b0b2-e4dfa0c5fecf nodeName:}" failed. No retries permitted until 2026-07-29 15:11:06.742607297 +0000 UTC m=+5.036544103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret") pod "global-pull-secret-syncer-vvz9k" (UID: "025b10db-422e-4bbc-b0b2-e4dfa0c5fecf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:04.843131 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:04.843092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:04.843352 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.843290    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:04.843352 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.843313    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:04.843352 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.843330    2560 projected.go:196] Error preparing data for projected volume kube-api-access-f7j65 for pod openshift-network-diagnostics/network-check-target-b7zwb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:04.843527 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:04.843394    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65 podName:43dda4c6-ff99-48e8-a28b-09250dfe09f1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:06.84337376 +0000 UTC m=+5.137310561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7j65" (UniqueName: "kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65") pod "network-check-target-b7zwb" (UID: "43dda4c6-ff99-48e8-a28b-09250dfe09f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:05.068852 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:05.068737    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 15:06:03 +0000 UTC" deadline="2028-01-26 15:34:35.664454009 +0000 UTC"
Jul 29 15:11:05.068852 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:05.068785    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13104h23m30.595674454s"
Jul 29 15:11:05.160272 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:05.160187    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:05.160471 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:05.160354    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:05.160471 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:05.160393    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:05.160596 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:05.160485    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:05.160596 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:05.160513    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:05.160596 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:05.160585    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:05.200157 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:05.200075    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dpmmq" event={"ID":"85c24726-0eb1-4ccf-83d9-9b4643142d10","Type":"ContainerStarted","Data":"94d20e300a2a98ec34eee0aacba6473ec1cd6bfb98a0b228424064d533b474a0"}
Jul 29 15:11:05.259008 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:05.258928    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:06.660384 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:06.660333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:06.660903 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.660502    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:06.660903 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.660573    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:10.660552264 +0000 UTC m=+8.954489052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:06.762295 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:06.761473    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:06.762295 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.761810    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:06.762295 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.761885    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret podName:025b10db-422e-4bbc-b0b2-e4dfa0c5fecf nodeName:}" failed. No retries permitted until 2026-07-29 15:11:10.761865026 +0000 UTC m=+9.055801827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret") pod "global-pull-secret-syncer-vvz9k" (UID: "025b10db-422e-4bbc-b0b2-e4dfa0c5fecf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:06.863093 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:06.862355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:06.863093 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.862585    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:06.863093 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.862606    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:06.863093 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.862618    2560 projected.go:196] Error preparing data for projected volume kube-api-access-f7j65 for pod openshift-network-diagnostics/network-check-target-b7zwb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:06.863093 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:06.862683    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65 podName:43dda4c6-ff99-48e8-a28b-09250dfe09f1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:10.862663819 +0000 UTC m=+9.156600621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7j65" (UniqueName: "kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65") pod "network-check-target-b7zwb" (UID: "43dda4c6-ff99-48e8-a28b-09250dfe09f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:07.160618 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:07.160582    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:07.160779 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:07.160735    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:07.161148 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:07.161130    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:07.161445 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:07.161251    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:07.161445 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:07.161318    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:07.161445 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:07.161388    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:09.159968 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:09.159933    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:09.160478 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:09.159952    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:09.160478 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:09.160066    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:09.160478 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:09.160157    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:09.160478 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:09.160259    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:09.160478 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:09.160334    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:10.696718 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:10.696625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:10.697172 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.696785    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:10.697172 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.696857    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:18.696837259 +0000 UTC m=+16.990774039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:10.797903 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:10.797306    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:10.797903 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.797506    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:10.797903 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.797566    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret podName:025b10db-422e-4bbc-b0b2-e4dfa0c5fecf nodeName:}" failed. No retries permitted until 2026-07-29 15:11:18.797549707 +0000 UTC m=+17.091486490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret") pod "global-pull-secret-syncer-vvz9k" (UID: "025b10db-422e-4bbc-b0b2-e4dfa0c5fecf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:10.898541 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:10.898503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:10.898734 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.898666    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:10.898734 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.898688    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:10.898734 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.898701    2560 projected.go:196] Error preparing data for projected volume kube-api-access-f7j65 for pod openshift-network-diagnostics/network-check-target-b7zwb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:10.898892 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:10.898766    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65 podName:43dda4c6-ff99-48e8-a28b-09250dfe09f1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:18.898747312 +0000 UTC m=+17.192684097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7j65" (UniqueName: "kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65") pod "network-check-target-b7zwb" (UID: "43dda4c6-ff99-48e8-a28b-09250dfe09f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:11.160379 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:11.160268    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:11.160537 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:11.160395    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:11.160537 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:11.160272    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:11.160537 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:11.160499    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:11.160537 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:11.160268    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:11.160743 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:11.160583    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:13.160860 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:13.160822    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:13.161377 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:13.160893    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:13.161377 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:13.161035    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:13.161377 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:13.161240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:13.161377 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:13.161330    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:13.161591 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:13.161401    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:15.160364 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:15.160293    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:15.160364 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:15.160338    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:15.160364 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:15.160338    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:15.160938 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:15.160427    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:15.160938 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:15.160514    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:15.160938 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:15.160615    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:17.159943 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:17.159906    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:17.160381 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:17.159915    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:17.160381 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:17.160038    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:17.160381 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:17.159916    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:17.160381 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:17.160121    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:17.160381 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:17.160201    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:18.754781 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:18.754747    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:18.755240 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.754870    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:18.755240 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.754925    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:34.754909259 +0000 UTC m=+33.048846039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:18.855817 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:18.855783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:18.855989 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.855930    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:18.855989 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.855982    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret podName:025b10db-422e-4bbc-b0b2-e4dfa0c5fecf nodeName:}" failed. No retries permitted until 2026-07-29 15:11:34.85596867 +0000 UTC m=+33.149905466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret") pod "global-pull-secret-syncer-vvz9k" (UID: "025b10db-422e-4bbc-b0b2-e4dfa0c5fecf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:18.957127 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:18.957082    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:18.957318 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.957282    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:18.957318 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.957306    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:18.957318 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.957317    2560 projected.go:196] Error preparing data for projected volume kube-api-access-f7j65 for pod openshift-network-diagnostics/network-check-target-b7zwb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:18.957447 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:18.957390    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65 podName:43dda4c6-ff99-48e8-a28b-09250dfe09f1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:34.95736975 +0000 UTC m=+33.251306547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7j65" (UniqueName: "kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65") pod "network-check-target-b7zwb" (UID: "43dda4c6-ff99-48e8-a28b-09250dfe09f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:19.160450 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:19.160419    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:19.160635 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:19.160534    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:19.160635 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:19.160543    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:19.160635 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:19.160565    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:19.160785 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:19.160652    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:19.160785 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:19.160721    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:21.160514 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:21.160478    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:21.160942 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:21.160478    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:21.160942 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:21.160478    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:21.160942 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:21.160629    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:21.160942 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:21.160768    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:21.160942 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:21.160881    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:21.961244 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:21.961219    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358e0c8d_7a8b_4876_9b32_2c379aeccecb.slice/crio-b30baf53feaba894aa5cf105410b260cb9cd92301250f01c41cad418676671b4 WatchSource:0}: Error finding container b30baf53feaba894aa5cf105410b260cb9cd92301250f01c41cad418676671b4: Status 404 returned error can't find the container with id b30baf53feaba894aa5cf105410b260cb9cd92301250f01c41cad418676671b4
Jul 29 15:11:22.237449 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:22.237417    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f2kcq" event={"ID":"0cbd1463-c4bc-44fb-8f42-c7e6dc518dc8","Type":"ContainerStarted","Data":"281c451874f421b209fed368dda92c8122533d2af710924ffab58c20a10b8707"}
Jul 29 15:11:22.239309 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:22.239282    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dpmmq" event={"ID":"85c24726-0eb1-4ccf-83d9-9b4643142d10","Type":"ContainerStarted","Data":"293273f07a913699de6ff0eb4217d59314c19d92c73d1f45bc6190c562c598d6"}
Jul 29 15:11:22.241643 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:22.241615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"a218a7e1ef96984135b6e5c0db121baa300cd9df394bec84270685cb964b0c6f"}
Jul 29 15:11:22.241747 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:22.241704    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"2c45cddd74ae3b12a3f1115c5cf5a5efecacf37e336788470adabe2d06c6dbcf"}
Jul 29 15:11:22.243151 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:22.243127    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vd2hl" event={"ID":"358e0c8d-7a8b-4876-9b32-2c379aeccecb","Type":"ContainerStarted","Data":"b30baf53feaba894aa5cf105410b260cb9cd92301250f01c41cad418676671b4"}
Jul 29 15:11:22.246372 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:22.246342    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal" event={"ID":"cf62266a6ae3b77620eda580b89764b9","Type":"ContainerStarted","Data":"b9efd317096c59319aca45bb78026d80da055cbd69d96effb4f20b22e348a826"}
Jul 29 15:11:22.260606 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:22.260373    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f2kcq" podStartSLOduration=2.044933852 podStartE2EDuration="20.260351038s" totalImagesPullingTime="18.215417186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:22.259833564 +0000 UTC m=+20.553770366" watchObservedRunningTime="2026-07-29 15:11:22.260351038 +0000 UTC m=+20.554287841"
Jul 29 15:11:23.161166 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.160966    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:23.161337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.160967    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:23.161337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.160977    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:23.161337 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:23.161295    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:23.161478 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:23.161390    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:23.161478 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:23.161450    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:23.251495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.251433    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:11:23.252297 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.251724    2560 generic.go:356] "Generic (PLEG): container finished" podID="c6e6ae73-f87c-4666-9d7b-28584bb70925" containerID="a218a7e1ef96984135b6e5c0db121baa300cd9df394bec84270685cb964b0c6f" exitCode=1
Jul 29 15:11:23.252297 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.251780    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerDied","Data":"a218a7e1ef96984135b6e5c0db121baa300cd9df394bec84270685cb964b0c6f"}
Jul 29 15:11:23.252297 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.251849    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"5f97c0cf1b10e44364810642d2146cd6c8da21d4fd3f0089b4fb2ee898c81322"}
Jul 29 15:11:23.252297 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.251881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"574febc99e4bbc0eaaf1f1e28a12327d2e287770e35314231ef2894308c3fec0"}
Jul 29 15:11:23.252297 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.251917    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"6dd7c2d938f0431e5fbfac4d9704ed788c54742ad7520a4abddc62fcb56c2da2"}
Jul 29 15:11:23.252297 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.251957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"76c2f2ff9088709e69da8534b30c10aa8529be5804d9e66a1683b1bb7746ccd3"}
Jul 29 15:11:23.253089 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.253066    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6b67r" event={"ID":"52224984-89d4-44bb-8119-2a95db6c1d03","Type":"ContainerStarted","Data":"1c94b166afd90edec973fc1a0ec025f3039187fb64ba8911a9f47da450c7e9c6"}
Jul 29 15:11:23.254487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.254462    2560 generic.go:356] "Generic (PLEG): container finished" podID="139e51abe32fc6d140a5f3ceeea4020f" containerID="b576489440a59a564068dc9b4e6e0cb2a20d05e13b867e39f2b36c1a567483d2" exitCode=0
Jul 29 15:11:23.254584 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.254538    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal" event={"ID":"139e51abe32fc6d140a5f3ceeea4020f","Type":"ContainerDied","Data":"b576489440a59a564068dc9b4e6e0cb2a20d05e13b867e39f2b36c1a567483d2"}
Jul 29 15:11:23.255981 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.255961    2560 generic.go:356] "Generic (PLEG): container finished" podID="eebdf1b2-5e28-4f3e-a36d-7687e491314d" containerID="568c37952a68cfff43c694b2000c492d31a1e64040c4fd1a8c65c14149407c45" exitCode=0
Jul 29 15:11:23.256063 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.256025    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerDied","Data":"568c37952a68cfff43c694b2000c492d31a1e64040c4fd1a8c65c14149407c45"}
Jul 29 15:11:23.257427 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.257400    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv" event={"ID":"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242","Type":"ContainerStarted","Data":"1bba050d86310a4fcde98b20f88af87ab2a2164ce46dfe15b67807e538f448bf"}
Jul 29 15:11:23.258915 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.258891    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6z2cx" event={"ID":"35b71621-959c-4601-9800-f5e3f26182fa","Type":"ContainerStarted","Data":"e4cda10d72bd7c7cf424fc158d6cc7dbfe676f494765e9c287c588e04db2407a"}
Jul 29 15:11:23.269165 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.269126    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6b67r" podStartSLOduration=3.296918326 podStartE2EDuration="21.269113102s" totalImagesPullingTime="17.972194776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:23.268688564 +0000 UTC m=+21.562625367" watchObservedRunningTime="2026-07-29 15:11:23.269113102 +0000 UTC m=+21.563049904"
Jul 29 15:11:23.287972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.287926    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dpmmq" podStartSLOduration=3.309610416 podStartE2EDuration="21.28790862s" totalImagesPullingTime="17.978298204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:23.287753075 +0000 UTC m=+21.581689877" watchObservedRunningTime="2026-07-29 15:11:23.28790862 +0000 UTC m=+21.581845422"
Jul 29 15:11:23.304784 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.304741    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6z2cx" podStartSLOduration=2.827485262 podStartE2EDuration="21.304726775s" totalImagesPullingTime="18.477241513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:23.304688147 +0000 UTC m=+21.598624952" watchObservedRunningTime="2026-07-29 15:11:23.304726775 +0000 UTC m=+21.598663576"
Jul 29 15:11:23.764932 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:23.764706    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 15:11:24.095343 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.095229    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T15:11:23.764920907Z","UUID":"7686e3a0-04a0-4986-9483-05175fb7cafb","Handler":null,"Name":"","Endpoint":""}
Jul 29 15:11:24.097502 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.096990    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 15:11:24.097502 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.097027    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 15:11:24.265891 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.265852    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv" event={"ID":"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242","Type":"ContainerStarted","Data":"26d7efa050777536758fc47bcc7162c8e1500baa6692552c7d9409357b763607"}
Jul 29 15:11:24.268116 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.268068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-64l8z" event={"ID":"49b10c1c-0978-4e3c-9145-d40da94ad15f","Type":"ContainerStarted","Data":"429918326a5e13b1bce6ac169d130d864759a170a579fb5b629b801a252ec08c"}
Jul 29 15:11:24.275365 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.275333    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal" event={"ID":"139e51abe32fc6d140a5f3ceeea4020f","Type":"ContainerStarted","Data":"2be48f27e9df731ae12998103409eef426b303bd22d9ca9cd9ff9cd39428a73e"}
Jul 29 15:11:24.278367 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.278337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vd2hl" event={"ID":"358e0c8d-7a8b-4876-9b32-2c379aeccecb","Type":"ContainerStarted","Data":"a135a228a41fe6b8c3ed48d2581bd3d30527ee640fe6ca23415b216702ab889b"}
Jul 29 15:11:24.284857 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.284812    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-64l8z" podStartSLOduration=3.870099348 podStartE2EDuration="22.284794707s" totalImagesPullingTime="18.414695359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:24.284685589 +0000 UTC m=+22.578622394" watchObservedRunningTime="2026-07-29 15:11:24.284794707 +0000 UTC m=+22.578731516"
Jul 29 15:11:24.322541 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.322484    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vd2hl" podStartSLOduration=20.629184693 podStartE2EDuration="22.322462778s" totalImagesPullingTime="1.693278085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:24.321707512 +0000 UTC m=+22.615644318" watchObservedRunningTime="2026-07-29 15:11:24.322462778 +0000 UTC m=+22.616399583"
Jul 29 15:11:24.322735 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.322708    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-199.ec2.internal" podStartSLOduration=21.322699278 podStartE2EDuration="21.322699278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:24.302559471 +0000 UTC m=+22.596496278" watchObservedRunningTime="2026-07-29 15:11:24.322699278 +0000 UTC m=+22.616636081"
Jul 29 15:11:24.649772 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.649743    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:24.650550 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:24.650506    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:25.160086 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:25.160056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:25.160399 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:25.160173    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:25.160399 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:25.160196    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:25.160399 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:25.160305    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:25.160399 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:25.160340    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:25.160624 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:25.160425    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:25.280966 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:25.280694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:25.281537 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:25.281500    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vd2hl"
Jul 29 15:11:26.285451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:26.285412    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv" event={"ID":"7a1ef0cc-f0c1-43b2-9cfb-93bd131b2242","Type":"ContainerStarted","Data":"47c6de92107a6a90f4be8bda6dd06d282e2fe56b621a7f4975bb500cd7fae567"}
Jul 29 15:11:26.289200 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:26.289157    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:11:26.289628 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:26.289601    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"9619835ab83dd12dc957a37758fd94adbfeb2724113b46b81eee0cda6bc1e9ff"}
Jul 29 15:11:26.291280 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:26.291255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal" event={"ID":"cf62266a6ae3b77620eda580b89764b9","Type":"ContainerStarted","Data":"ad39432e934ede0117c84909a3b04515098c5eb8134af52d04456c2b6f4fa48f"}
Jul 29 15:11:26.309806 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:26.309755    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dx2lv" podStartSLOduration=3.140162551 podStartE2EDuration="24.309737359s" totalImagesPullingTime="21.169574808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:26.309403521 +0000 UTC m=+24.603340324" watchObservedRunningTime="2026-07-29 15:11:26.309737359 +0000 UTC m=+24.603674162"
Jul 29 15:11:26.324236 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:26.324173    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-199.ec2.internal" podStartSLOduration=23.324155192 podStartE2EDuration="23.324155192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:26.323935982 +0000 UTC m=+24.617872796" watchObservedRunningTime="2026-07-29 15:11:26.324155192 +0000 UTC m=+24.618091995"
Jul 29 15:11:27.160736 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:27.160697    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:27.160924 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:27.160697    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:27.160924 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:27.160815    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:27.160924 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:27.160913    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:27.161120 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:27.160700    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:27.161120 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:27.161025    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:28.298916 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:28.298734    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:11:28.299513 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:28.299266    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"8c8405f9c4f16ddfd1b523dab463b3325fcb63664ec1212a557f4dd266a429ab"}
Jul 29 15:11:28.299589 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:28.299510    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:28.299730 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:28.299713    2560 scope.go:117] "RemoveContainer" containerID="a218a7e1ef96984135b6e5c0db121baa300cd9df394bec84270685cb964b0c6f"
Jul 29 15:11:28.304045 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:28.304014    2560 generic.go:356] "Generic (PLEG): container finished" podID="eebdf1b2-5e28-4f3e-a36d-7687e491314d" containerID="881f4025c38e509c9cf1c96c80ee1125467ce829fb289a1944c453099cc281c7" exitCode=0
Jul 29 15:11:28.304201 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:28.304074    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerDied","Data":"881f4025c38e509c9cf1c96c80ee1125467ce829fb289a1944c453099cc281c7"}
Jul 29 15:11:28.316860 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:28.316758    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:29.159944 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.159907    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:29.160162 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.159977    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:29.160162 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.160062    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:29.160162 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:29.160078    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:29.160379 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:29.160170    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:29.160379 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:29.160255    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:29.310431 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.310406    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:11:29.310977 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.310774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" event={"ID":"c6e6ae73-f87c-4666-9d7b-28584bb70925","Type":"ContainerStarted","Data":"b79f13b47cedb5e74f66182d01c4bcb061ddce685f31496098dfbffc8040a5d3"}
Jul 29 15:11:29.311206 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.311169    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:29.311356 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.311258    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:29.326178 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.326151    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:11:29.342551 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:29.342505    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw" podStartSLOduration=9.127120944 podStartE2EDuration="27.342489971s" totalImagesPullingTime="18.215369027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:29.340997408 +0000 UTC m=+27.634934220" watchObservedRunningTime="2026-07-29 15:11:29.342489971 +0000 UTC m=+27.636426772"
Jul 29 15:11:30.315431 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.315352    2560 generic.go:356] "Generic (PLEG): container finished" podID="eebdf1b2-5e28-4f3e-a36d-7687e491314d" containerID="5286ad9dedfa0a857b454c4bf845edc57cee3aeeedc2808cd1cdace7b43a5989" exitCode=0
Jul 29 15:11:30.315884 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.315435    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerDied","Data":"5286ad9dedfa0a857b454c4bf845edc57cee3aeeedc2808cd1cdace7b43a5989"}
Jul 29 15:11:30.442538 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.442493    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vvz9k"]
Jul 29 15:11:30.442884 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.442865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:30.443016 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:30.442977    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:30.443745 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.443421    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gnjvk"]
Jul 29 15:11:30.443745 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.443535    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:30.443745 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:30.443632    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:30.444392 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.444370    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b7zwb"]
Jul 29 15:11:30.444500 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:30.444464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:30.444564 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:30.444546    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:32.163133 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:32.162930    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:32.163627 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:32.162969    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:32.163627 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:32.163223    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:32.163627 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:32.162980    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:32.163627 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:32.163275    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:32.163627 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:32.163390    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:32.322566 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:32.322536    2560 generic.go:356] "Generic (PLEG): container finished" podID="eebdf1b2-5e28-4f3e-a36d-7687e491314d" containerID="0166fadb82572f51bd6c9c6bdf0b40c29f60d1c27cf4abd294a73087577985a7" exitCode=0
Jul 29 15:11:32.322760 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:32.322594    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerDied","Data":"0166fadb82572f51bd6c9c6bdf0b40c29f60d1c27cf4abd294a73087577985a7"}
Jul 29 15:11:34.160327 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:34.160292    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:34.160796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:34.160429    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:34.160796 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.160439    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnjvk" podUID="2415e25c-6267-4eb0-bb24-11872c713139"
Jul 29 15:11:34.160796 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.160539    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7zwb" podUID="43dda4c6-ff99-48e8-a28b-09250dfe09f1"
Jul 29 15:11:34.160796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:34.160588    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:34.160796 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.160663    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vvz9k" podUID="025b10db-422e-4bbc-b0b2-e4dfa0c5fecf"
Jul 29 15:11:34.770609 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:34.770563    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:34.770795 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.770721    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:34.770855 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.770797    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:06.770775993 +0000 UTC m=+65.064712775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:34.871917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:34.871831    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:34.872080 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.872016    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:34.872139 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.872087    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret podName:025b10db-422e-4bbc-b0b2-e4dfa0c5fecf nodeName:}" failed. No retries permitted until 2026-07-29 15:12:06.872072102 +0000 UTC m=+65.166008882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret") pod "global-pull-secret-syncer-vvz9k" (UID: "025b10db-422e-4bbc-b0b2-e4dfa0c5fecf") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:34.973208 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:34.973151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:34.973385 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.973362    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:34.973465 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.973388    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:34.973465 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.973402    2560 projected.go:196] Error preparing data for projected volume kube-api-access-f7j65 for pod openshift-network-diagnostics/network-check-target-b7zwb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:34.973553 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:34.973472    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65 podName:43dda4c6-ff99-48e8-a28b-09250dfe09f1 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:06.973451323 +0000 UTC m=+65.267388105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7j65" (UniqueName: "kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65") pod "network-check-target-b7zwb" (UID: "43dda4c6-ff99-48e8-a28b-09250dfe09f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:35.100758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.100727    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-199.ec2.internal" event="NodeReady"
Jul 29 15:11:35.100924 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.100884    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 15:11:35.146082 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.145959    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d6d695b6d-9gwgt"]
Jul 29 15:11:35.149585 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.149562    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.152000 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.151976    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-snrjc\""
Jul 29 15:11:35.152107 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.151978    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 15:11:35.152314 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.152293    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 15:11:35.152806 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.152788    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 15:11:35.157803 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.157470    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 15:11:35.164015 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.163991    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d6d695b6d-9gwgt"]
Jul 29 15:11:35.169556 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.169531    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dpl5l"]
Jul 29 15:11:35.173341 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.173319    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h79ts"]
Jul 29 15:11:35.173476 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.173457    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.176037 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.175982    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2gcqd\""
Jul 29 15:11:35.176037 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.175981    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 15:11:35.176249 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.176009    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 15:11:35.176417 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.176402    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 15:11:35.177048 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.177029    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.178996 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.178977    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 15:11:35.180259 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.180242    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw7hg\""
Jul 29 15:11:35.180364 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.180245    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 15:11:35.189883 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.189856    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dpl5l"]
Jul 29 15:11:35.190250 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.190229    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h79ts"]
Jul 29 15:11:35.275015 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.274967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.275015 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6hcm\" (UniqueName: \"kubernetes.io/projected/c46543f0-2496-4e4d-9a40-4cc67663aedf-kube-api-access-c6hcm\") pod \"ingress-canary-dpl5l\" (UID: \"c46543f0-2496-4e4d-9a40-4cc67663aedf\") " pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.275318 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdf8ca9f-0611-463b-b647-e7809c88f8c1-tmp-dir\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.275318 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275070    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-trusted-ca\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.275318 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-installation-pull-secrets\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.275318 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-bound-sa-token\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.275318 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-image-registry-private-configuration\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.275318 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5nhp\" (UniqueName: \"kubernetes.io/projected/bdf8ca9f-0611-463b-b647-e7809c88f8c1-kube-api-access-b5nhp\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.275536 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275340    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.275536 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275367    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-certificates\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.275536 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c46543f0-2496-4e4d-9a40-4cc67663aedf-cert\") pod \"ingress-canary-dpl5l\" (UID: \"c46543f0-2496-4e4d-9a40-4cc67663aedf\") " pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.275536 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275449    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xwxw\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-kube-api-access-6xwxw\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.275536 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275479    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf8ca9f-0611-463b-b647-e7809c88f8c1-config-volume\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.275536 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.275518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-ca-trust-extracted\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.367547 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.367514    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vd2hl_358e0c8d-7a8b-4876-9b32-2c379aeccecb/konnectivity-agent/0.log" limit=1024
Jul 29 15:11:35.376565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-image-registry-private-configuration\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.376753 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376593    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5nhp\" (UniqueName: \"kubernetes.io/projected/bdf8ca9f-0611-463b-b647-e7809c88f8c1-kube-api-access-b5nhp\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.376753 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.376753 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-certificates\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.376753 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376743    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c46543f0-2496-4e4d-9a40-4cc67663aedf-cert\") pod \"ingress-canary-dpl5l\" (UID: \"c46543f0-2496-4e4d-9a40-4cc67663aedf\") " pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xwxw\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-kube-api-access-6xwxw\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.376786    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376808    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf8ca9f-0611-463b-b647-e7809c88f8c1-config-volume\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.376861    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls podName:bdf8ca9f-0611-463b-b647-e7809c88f8c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:35.87683922 +0000 UTC m=+34.170776019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls") pod "dns-default-h79ts" (UID: "bdf8ca9f-0611-463b-b647-e7809c88f8c1") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376883    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-ca-trust-extracted\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6hcm\" (UniqueName: \"kubernetes.io/projected/c46543f0-2496-4e4d-9a40-4cc67663aedf-kube-api-access-c6hcm\") pod \"ingress-canary-dpl5l\" (UID: \"c46543f0-2496-4e4d-9a40-4cc67663aedf\") " pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.376972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdf8ca9f-0611-463b-b647-e7809c88f8c1-tmp-dir\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.377356 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.376993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-trusted-ca\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.377356 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.377020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-installation-pull-secrets\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.377356 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.377046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-bound-sa-token\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.377356 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.377260    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-certificates\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.377547 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.377378    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf8ca9f-0611-463b-b647-e7809c88f8c1-config-volume\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.377547 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.377443    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:11:35.377547 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.377460    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d6d695b6d-9gwgt: secret "image-registry-tls" not found
Jul 29 15:11:35.377547 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.377524    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls podName:bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab nodeName:}" failed. No retries permitted until 2026-07-29 15:11:35.877504841 +0000 UTC m=+34.171441641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls") pod "image-registry-d6d695b6d-9gwgt" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab") : secret "image-registry-tls" not found
Jul 29 15:11:35.377743 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.377633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdf8ca9f-0611-463b-b647-e7809c88f8c1-tmp-dir\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.377743 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.377665    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-ca-trust-extracted\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.378272 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.378177    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-trusted-ca\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.382117 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.382045    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-image-registry-private-configuration\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.382117 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.382060    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c46543f0-2496-4e4d-9a40-4cc67663aedf-cert\") pod \"ingress-canary-dpl5l\" (UID: \"c46543f0-2496-4e4d-9a40-4cc67663aedf\") " pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.382117 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.382064    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-installation-pull-secrets\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.392942 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.392843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6hcm\" (UniqueName: \"kubernetes.io/projected/c46543f0-2496-4e4d-9a40-4cc67663aedf-kube-api-access-c6hcm\") pod \"ingress-canary-dpl5l\" (UID: \"c46543f0-2496-4e4d-9a40-4cc67663aedf\") " pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.392942 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.392889    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-bound-sa-token\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.393116 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.393052    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5nhp\" (UniqueName: \"kubernetes.io/projected/bdf8ca9f-0611-463b-b647-e7809c88f8c1-kube-api-access-b5nhp\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.393373 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.393356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xwxw\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-kube-api-access-6xwxw\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.468393 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.468304    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vd2hl_358e0c8d-7a8b-4876-9b32-2c379aeccecb/konnectivity-agent/0.log" limit=1024
Jul 29 15:11:35.485270 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.485231    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dpl5l"
Jul 29 15:11:35.587104 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.587067    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vd2hl_358e0c8d-7a8b-4876-9b32-2c379aeccecb/konnectivity-agent/0.log" limit=1024
Jul 29 15:11:35.668036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.667997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vd2hl_358e0c8d-7a8b-4876-9b32-2c379aeccecb/konnectivity-agent/0.log" limit=1024
Jul 29 15:11:35.880824 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.880786    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:35.881023 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:35.880863    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:35.881023 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.880979    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:11:35.881023 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.880994    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d6d695b6d-9gwgt: secret "image-registry-tls" not found
Jul 29 15:11:35.881173 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.880980    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:35.881173 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.881066    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls podName:bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab nodeName:}" failed. No retries permitted until 2026-07-29 15:11:36.881041816 +0000 UTC m=+35.174978615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls") pod "image-registry-d6d695b6d-9gwgt" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab") : secret "image-registry-tls" not found
Jul 29 15:11:35.881173 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:35.881164    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls podName:bdf8ca9f-0611-463b-b647-e7809c88f8c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:36.881142406 +0000 UTC m=+35.175079188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls") pod "dns-default-h79ts" (UID: "bdf8ca9f-0611-463b-b647-e7809c88f8c1") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:36.163989 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.163908    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:11:36.163989 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.163933    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:11:36.164526 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.164254    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:11:36.166839 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.166802    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cnmln\""
Jul 29 15:11:36.167712 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.167693    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 15:11:36.167955 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.167934    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 15:11:36.168062 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.168040    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 15:11:36.168198 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.168082    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sgb7s\""
Jul 29 15:11:36.168198 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.168106    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 15:11:36.393578 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.393416    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dpl5l"]
Jul 29 15:11:36.397137 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:11:36.397101    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46543f0_2496_4e4d_9a40_4cc67663aedf.slice/crio-45e0329959ebbafb020eac3146baa990b41dd135b9d93ca0144bf7d87f194e9e WatchSource:0}: Error finding container 45e0329959ebbafb020eac3146baa990b41dd135b9d93ca0144bf7d87f194e9e: Status 404 returned error can't find the container with id 45e0329959ebbafb020eac3146baa990b41dd135b9d93ca0144bf7d87f194e9e
Jul 29 15:11:36.888347 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.888301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:36.888529 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:36.888378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:36.888529 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:36.888457    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:36.888529 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:36.888528    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls podName:bdf8ca9f-0611-463b-b647-e7809c88f8c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:38.888512744 +0000 UTC m=+37.182449525 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls") pod "dns-default-h79ts" (UID: "bdf8ca9f-0611-463b-b647-e7809c88f8c1") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:36.888640 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:36.888460    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:11:36.888640 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:36.888562    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d6d695b6d-9gwgt: secret "image-registry-tls" not found
Jul 29 15:11:36.888640 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:36.888607    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls podName:bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab nodeName:}" failed. No retries permitted until 2026-07-29 15:11:38.888595699 +0000 UTC m=+37.182532502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls") pod "image-registry-d6d695b6d-9gwgt" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab") : secret "image-registry-tls" not found
Jul 29 15:11:37.338370 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:37.338287    2560 generic.go:356] "Generic (PLEG): container finished" podID="eebdf1b2-5e28-4f3e-a36d-7687e491314d" containerID="9269c84fa1badb1c0777ea2494e8bdae6d1dd68e940f33ec7e01513f981f9cb0" exitCode=0
Jul 29 15:11:37.339239 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:37.338377    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerDied","Data":"9269c84fa1badb1c0777ea2494e8bdae6d1dd68e940f33ec7e01513f981f9cb0"}
Jul 29 15:11:37.339527 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:37.339447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dpl5l" event={"ID":"c46543f0-2496-4e4d-9a40-4cc67663aedf","Type":"ContainerStarted","Data":"45e0329959ebbafb020eac3146baa990b41dd135b9d93ca0144bf7d87f194e9e"}
Jul 29 15:11:38.347043 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:38.346810    2560 generic.go:356] "Generic (PLEG): container finished" podID="eebdf1b2-5e28-4f3e-a36d-7687e491314d" containerID="3fb74d329c96b20cd82017529423fa23f4a3c42fc4df5cff7084fc9dbf008ee6" exitCode=0
Jul 29 15:11:38.347405 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:38.346886    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerDied","Data":"3fb74d329c96b20cd82017529423fa23f4a3c42fc4df5cff7084fc9dbf008ee6"}
Jul 29 15:11:38.904700 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:38.904657    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:38.904867 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:38.904761    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:38.904867 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:38.904852    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:11:38.904950 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:38.904878    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d6d695b6d-9gwgt: secret "image-registry-tls" not found
Jul 29 15:11:38.904950 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:38.904892    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:38.904950 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:38.904949    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls podName:bdf8ca9f-0611-463b-b647-e7809c88f8c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:42.904931622 +0000 UTC m=+41.198868421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls") pod "dns-default-h79ts" (UID: "bdf8ca9f-0611-463b-b647-e7809c88f8c1") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:38.905169 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:38.904967    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls podName:bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab nodeName:}" failed. No retries permitted until 2026-07-29 15:11:42.904958889 +0000 UTC m=+41.198895682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls") pod "image-registry-d6d695b6d-9gwgt" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab") : secret "image-registry-tls" not found
Jul 29 15:11:39.352803 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:39.352771    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" event={"ID":"eebdf1b2-5e28-4f3e-a36d-7687e491314d","Type":"ContainerStarted","Data":"51fde70d569b208355c8ed8650d795f716c70589eb52e1cfe688c4f217879bae"}
Jul 29 15:11:39.383296 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:39.383230    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tlxfp" podStartSLOduration=8.848314182 podStartE2EDuration="37.38320842s" totalImagesPullingTime="28.534894238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:39.382942834 +0000 UTC m=+37.676879638" watchObservedRunningTime="2026-07-29 15:11:39.38320842 +0000 UTC m=+37.677145216"
Jul 29 15:11:41.358659 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:41.358619    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dpl5l" event={"ID":"c46543f0-2496-4e4d-9a40-4cc67663aedf","Type":"ContainerStarted","Data":"ba63169ab5c53202606d18909838c89bacad2835007037ea4ab3bbc1e5a8fe3e"}
Jul 29 15:11:41.405157 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:41.405095    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dpl5l" podStartSLOduration=2.032401007 podStartE2EDuration="6.405074501s" totalImagesPullingTime="4.372673494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:41.404098347 +0000 UTC m=+39.698035179" watchObservedRunningTime="2026-07-29 15:11:41.405074501 +0000 UTC m=+39.699011305"
Jul 29 15:11:42.935644 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:42.935610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:42.936029 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:42.935676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:42.936029 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:42.935767    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:11:42.936029 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:42.935786    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d6d695b6d-9gwgt: secret "image-registry-tls" not found
Jul 29 15:11:42.936029 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:42.935793    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:42.936029 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:42.935836    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls podName:bdf8ca9f-0611-463b-b647-e7809c88f8c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:50.935822785 +0000 UTC m=+49.229759566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls") pod "dns-default-h79ts" (UID: "bdf8ca9f-0611-463b-b647-e7809c88f8c1") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:42.936029 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:42.935853    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls podName:bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab nodeName:}" failed. No retries permitted until 2026-07-29 15:11:50.935844742 +0000 UTC m=+49.229781522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls") pod "image-registry-d6d695b6d-9gwgt" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab") : secret "image-registry-tls" not found
Jul 29 15:11:50.994146 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:50.994095    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:11:50.994648 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:11:50.994178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:11:50.994648 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:50.994267    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:50.994648 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:50.994283    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:11:50.994648 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:50.994298    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d6d695b6d-9gwgt: secret "image-registry-tls" not found
Jul 29 15:11:50.994648 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:50.994332    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls podName:bdf8ca9f-0611-463b-b647-e7809c88f8c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:06.994316694 +0000 UTC m=+65.288253474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls") pod "dns-default-h79ts" (UID: "bdf8ca9f-0611-463b-b647-e7809c88f8c1") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:50.994648 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:11:50.994347    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls podName:bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab nodeName:}" failed. No retries permitted until 2026-07-29 15:12:06.994341129 +0000 UTC m=+65.288277909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls") pod "image-registry-d6d695b6d-9gwgt" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab") : secret "image-registry-tls" not found
Jul 29 15:12:01.327881 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:01.327845    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sscmw"
Jul 29 15:12:06.817259 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:06.817207    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:12:06.820039 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:06.820015    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 15:12:06.828291 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:06.828262    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 15:12:06.828408 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:06.828359    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs podName:2415e25c-6267-4eb0-bb24-11872c713139 nodeName:}" failed. No retries permitted until 2026-07-29 15:13:10.82833471 +0000 UTC m=+129.122271492 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs") pod "network-metrics-daemon-gnjvk" (UID: "2415e25c-6267-4eb0-bb24-11872c713139") : secret "metrics-daemon-secret" not found
Jul 29 15:12:06.917980 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:06.917942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:12:06.920546 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:06.920524    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 15:12:06.932392 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:06.932361    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/025b10db-422e-4bbc-b0b2-e4dfa0c5fecf-original-pull-secret\") pod \"global-pull-secret-syncer-vvz9k\" (UID: \"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf\") " pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:12:07.019124 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.019093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:12:07.019349 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.019140    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:12:07.019349 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.019162    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:12:07.019349 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:07.019286    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:12:07.019349 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:07.019298    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d6d695b6d-9gwgt: secret "image-registry-tls" not found
Jul 29 15:12:07.019349 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:07.019341    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:12:07.019594 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:07.019357    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls podName:bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab nodeName:}" failed. No retries permitted until 2026-07-29 15:12:39.019341688 +0000 UTC m=+97.313278467 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls") pod "image-registry-d6d695b6d-9gwgt" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab") : secret "image-registry-tls" not found
Jul 29 15:12:07.019594 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:07.019412    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls podName:bdf8ca9f-0611-463b-b647-e7809c88f8c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:39.019393117 +0000 UTC m=+97.313329908 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls") pod "dns-default-h79ts" (UID: "bdf8ca9f-0611-463b-b647-e7809c88f8c1") : secret "dns-default-metrics-tls" not found
Jul 29 15:12:07.021797 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.021779    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 15:12:07.031686 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.031666    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 15:12:07.042994 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.042970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7j65\" (UniqueName: \"kubernetes.io/projected/43dda4c6-ff99-48e8-a28b-09250dfe09f1-kube-api-access-f7j65\") pod \"network-check-target-b7zwb\" (UID: \"43dda4c6-ff99-48e8-a28b-09250dfe09f1\") " pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:12:07.079629 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.079551    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cnmln\""
Jul 29 15:12:07.087816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.087792    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:12:07.091110 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.091089    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vvz9k"
Jul 29 15:12:07.229714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.229686    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vvz9k"]
Jul 29 15:12:07.232719 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:07.232694    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025b10db_422e_4bbc_b0b2_e4dfa0c5fecf.slice/crio-8a842c5a50d6b6cf91d6e1b301981797edbdfa4289d8b12dad5890c7dfce148f WatchSource:0}: Error finding container 8a842c5a50d6b6cf91d6e1b301981797edbdfa4289d8b12dad5890c7dfce148f: Status 404 returned error can't find the container with id 8a842c5a50d6b6cf91d6e1b301981797edbdfa4289d8b12dad5890c7dfce148f
Jul 29 15:12:07.242863 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.242837    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b7zwb"]
Jul 29 15:12:07.245857 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:07.245828    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43dda4c6_ff99_48e8_a28b_09250dfe09f1.slice/crio-b800c6bca20e4be9fa33455c67c9dfb197b898bfa74fd8b455b907279116e642 WatchSource:0}: Error finding container b800c6bca20e4be9fa33455c67c9dfb197b898bfa74fd8b455b907279116e642: Status 404 returned error can't find the container with id b800c6bca20e4be9fa33455c67c9dfb197b898bfa74fd8b455b907279116e642
Jul 29 15:12:07.419470 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.419435    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b7zwb" event={"ID":"43dda4c6-ff99-48e8-a28b-09250dfe09f1","Type":"ContainerStarted","Data":"b800c6bca20e4be9fa33455c67c9dfb197b898bfa74fd8b455b907279116e642"}
Jul 29 15:12:07.420479 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:07.420451    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vvz9k" event={"ID":"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf","Type":"ContainerStarted","Data":"8a842c5a50d6b6cf91d6e1b301981797edbdfa4289d8b12dad5890c7dfce148f"}
Jul 29 15:12:10.431025 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:10.430986    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b7zwb" event={"ID":"43dda4c6-ff99-48e8-a28b-09250dfe09f1","Type":"ContainerStarted","Data":"9a11684cc1805b6f26c40e81d62c8f62b26c81835f7a5fb206c38120e4295b52"}
Jul 29 15:12:10.431481 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:10.431114    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:12:14.005151 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.005104    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b7zwb" podStartSLOduration=69.312053029 podStartE2EDuration="1m12.005087543s" totalImagesPullingTime="2.693034514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:10.447618757 +0000 UTC m=+68.741555559" watchObservedRunningTime="2026-07-29 15:12:14.005087543 +0000 UTC m=+72.299024345"
Jul 29 15:12:14.005796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.005774    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"]
Jul 29 15:12:14.008888 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.008870    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:14.018493 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.017204    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"]
Jul 29 15:12:14.019921 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.019383    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 29 15:12:14.019921 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.019668    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6h7pv\""
Jul 29 15:12:14.019921 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.019867    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 29 15:12:14.020582 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.020564    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"
Jul 29 15:12:14.026748 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.026730    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"]
Jul 29 15:12:14.027194 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.027147    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:12:14.027194 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.027147    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7jr9g\""
Jul 29 15:12:14.027467 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.027453    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 15:12:14.029360 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.029345    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.032534 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.032507    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 15:12:14.033027 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.033012    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 15:12:14.033111 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.033041    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 15:12:14.033416 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.033402    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pg658\""
Jul 29 15:12:14.033647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.033632    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:12:14.046626 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.046598    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"]
Jul 29 15:12:14.047624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.047603    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"]
Jul 29 15:12:14.053790 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.053770    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"]
Jul 29 15:12:14.066252 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.066231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/96198947-8a41-49ad-baeb-c8d609ef2cb8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.066355 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.066286    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/96198947-8a41-49ad-baeb-c8d609ef2cb8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.066355 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.066305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6875b039-7cc3-425b-a5aa-c45f0d63d01a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:14.066430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.066376    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6flpc\" (UniqueName: \"kubernetes.io/projected/96198947-8a41-49ad-baeb-c8d609ef2cb8-kube-api-access-6flpc\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.066430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.066408    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:14.066430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.066426    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44w7\" (UniqueName: \"kubernetes.io/projected/a6fbfe9b-c402-42c5-a0c1-258a123a5821-kube-api-access-v44w7\") pod \"volume-data-source-validator-549c6d54cf-ctnd2\" (UID: \"a6fbfe9b-c402-42c5-a0c1-258a123a5821\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"
Jul 29 15:12:14.119799 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.119765    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"]
Jul 29 15:12:14.122567 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.122552    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.126612 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.126590    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 29 15:12:14.126756 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.126630    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 29 15:12:14.126756 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.126630    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 15:12:14.126756 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.126674    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 15:12:14.126756 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.126699    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zsml9\""
Jul 29 15:12:14.142963 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.142938    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"]
Jul 29 15:12:14.167123 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/96198947-8a41-49ad-baeb-c8d609ef2cb8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.167248 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6875b039-7cc3-425b-a5aa-c45f0d63d01a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:14.167248 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnjqz\" (UniqueName: \"kubernetes.io/projected/d8b85fba-8719-4dee-a1b5-e94d9747836b-kube-api-access-jnjqz\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.167248 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167171    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6flpc\" (UniqueName: \"kubernetes.io/projected/96198947-8a41-49ad-baeb-c8d609ef2cb8-kube-api-access-6flpc\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.167248 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167206    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:14.167248 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44w7\" (UniqueName: \"kubernetes.io/projected/a6fbfe9b-c402-42c5-a0c1-258a123a5821-kube-api-access-v44w7\") pod \"volume-data-source-validator-549c6d54cf-ctnd2\" (UID: \"a6fbfe9b-c402-42c5-a0c1-258a123a5821\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"
Jul 29 15:12:14.167467 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.167467 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.167360    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 15:12:14.167566 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d8b85fba-8719-4dee-a1b5-e94d9747836b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.167566 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.167500    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert podName:6875b039-7cc3-425b-a5aa-c45f0d63d01a nodeName:}" failed. No retries permitted until 2026-07-29 15:12:14.667479568 +0000 UTC m=+72.961416359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rrds9" (UID: "6875b039-7cc3-425b-a5aa-c45f0d63d01a") : secret "networking-console-plugin-cert" not found
Jul 29 15:12:14.167566 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167548    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/96198947-8a41-49ad-baeb-c8d609ef2cb8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.167719 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/96198947-8a41-49ad-baeb-c8d609ef2cb8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.167835 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.167821    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6875b039-7cc3-425b-a5aa-c45f0d63d01a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:14.169638 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.169617    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/96198947-8a41-49ad-baeb-c8d609ef2cb8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.189022 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.188987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44w7\" (UniqueName: \"kubernetes.io/projected/a6fbfe9b-c402-42c5-a0c1-258a123a5821-kube-api-access-v44w7\") pod \"volume-data-source-validator-549c6d54cf-ctnd2\" (UID: \"a6fbfe9b-c402-42c5-a0c1-258a123a5821\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"
Jul 29 15:12:14.190756 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.190736    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6flpc\" (UniqueName: \"kubernetes.io/projected/96198947-8a41-49ad-baeb-c8d609ef2cb8-kube-api-access-6flpc\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5\" (UID: \"96198947-8a41-49ad-baeb-c8d609ef2cb8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.268205 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.268096    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d8b85fba-8719-4dee-a1b5-e94d9747836b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.268205 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.268197    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnjqz\" (UniqueName: \"kubernetes.io/projected/d8b85fba-8719-4dee-a1b5-e94d9747836b-kube-api-access-jnjqz\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.268445 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.268227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.268445 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.268322    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:14.268445 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.268387    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls podName:d8b85fba-8719-4dee-a1b5-e94d9747836b nodeName:}" failed. No retries permitted until 2026-07-29 15:12:14.768372773 +0000 UTC m=+73.062309553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ccv7r" (UID: "d8b85fba-8719-4dee-a1b5-e94d9747836b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:14.268879 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.268852    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d8b85fba-8719-4dee-a1b5-e94d9747836b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.287307 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.287279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnjqz\" (UniqueName: \"kubernetes.io/projected/d8b85fba-8719-4dee-a1b5-e94d9747836b-kube-api-access-jnjqz\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.328196 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.328150    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"
Jul 29 15:12:14.337125 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.337101    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"
Jul 29 15:12:14.445057 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.445022    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vvz9k" event={"ID":"025b10db-422e-4bbc-b0b2-e4dfa0c5fecf","Type":"ContainerStarted","Data":"a10c8a30e843ae8e7d36ae5e6098ea35542a3a8bccf0d9ca96ceb20c2e3a51c3"}
Jul 29 15:12:14.456313 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.456294    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2"]
Jul 29 15:12:14.458051 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:14.458021    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6fbfe9b_c402_42c5_a0c1_258a123a5821.slice/crio-7176a51f872c85b54ac46d763cb120d09a6267f6f33cd5a216bcc3bbc1e10744 WatchSource:0}: Error finding container 7176a51f872c85b54ac46d763cb120d09a6267f6f33cd5a216bcc3bbc1e10744: Status 404 returned error can't find the container with id 7176a51f872c85b54ac46d763cb120d09a6267f6f33cd5a216bcc3bbc1e10744
Jul 29 15:12:14.471527 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.471485    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vvz9k" podStartSLOduration=65.131125484 podStartE2EDuration="1m11.471471096s" totalImagesPullingTime="6.340345612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:14.470691581 +0000 UTC m=+72.764628383" watchObservedRunningTime="2026-07-29 15:12:14.471471096 +0000 UTC m=+72.765407898"
Jul 29 15:12:14.473133 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.473113    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5"]
Jul 29 15:12:14.475178 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:14.475155    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96198947_8a41_49ad_baeb_c8d609ef2cb8.slice/crio-a9a452df647d832f074c0f8625e586961a685a872d44cf146583cc259dd9c633 WatchSource:0}: Error finding container a9a452df647d832f074c0f8625e586961a685a872d44cf146583cc259dd9c633: Status 404 returned error can't find the container with id a9a452df647d832f074c0f8625e586961a685a872d44cf146583cc259dd9c633
Jul 29 15:12:14.672358 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.672319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:14.672521 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.672469    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 15:12:14.672563 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.672534    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert podName:6875b039-7cc3-425b-a5aa-c45f0d63d01a nodeName:}" failed. No retries permitted until 2026-07-29 15:12:15.672518977 +0000 UTC m=+73.966455756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rrds9" (UID: "6875b039-7cc3-425b-a5aa-c45f0d63d01a") : secret "networking-console-plugin-cert" not found
Jul 29 15:12:14.772962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:14.772920    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:14.773128 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.773068    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:14.773173 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:14.773130    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls podName:d8b85fba-8719-4dee-a1b5-e94d9747836b nodeName:}" failed. No retries permitted until 2026-07-29 15:12:15.773116216 +0000 UTC m=+74.067052996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ccv7r" (UID: "d8b85fba-8719-4dee-a1b5-e94d9747836b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:15.449733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:15.449690    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2" event={"ID":"a6fbfe9b-c402-42c5-a0c1-258a123a5821","Type":"ContainerStarted","Data":"7176a51f872c85b54ac46d763cb120d09a6267f6f33cd5a216bcc3bbc1e10744"}
Jul 29 15:12:15.451106 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:15.451071    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5" event={"ID":"96198947-8a41-49ad-baeb-c8d609ef2cb8","Type":"ContainerStarted","Data":"a9a452df647d832f074c0f8625e586961a685a872d44cf146583cc259dd9c633"}
Jul 29 15:12:15.681800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:15.681763    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:15.681962 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:15.681916    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 15:12:15.682008 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:15.681984    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert podName:6875b039-7cc3-425b-a5aa-c45f0d63d01a nodeName:}" failed. No retries permitted until 2026-07-29 15:12:17.681968985 +0000 UTC m=+75.975905769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rrds9" (UID: "6875b039-7cc3-425b-a5aa-c45f0d63d01a") : secret "networking-console-plugin-cert" not found
Jul 29 15:12:15.783346 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:15.783246    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:15.783511 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:15.783421    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:15.783578 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:15.783536    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls podName:d8b85fba-8719-4dee-a1b5-e94d9747836b nodeName:}" failed. No retries permitted until 2026-07-29 15:12:17.783516337 +0000 UTC m=+76.077453118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ccv7r" (UID: "d8b85fba-8719-4dee-a1b5-e94d9747836b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:16.455984 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:16.455943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2" event={"ID":"a6fbfe9b-c402-42c5-a0c1-258a123a5821","Type":"ContainerStarted","Data":"ee25f31941ca0bd2e0991c34b1434237dcde53f357f473489bfe2c208d23b042"}
Jul 29 15:12:16.476076 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:16.476018    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ctnd2" podStartSLOduration=1.906885172 podStartE2EDuration="3.476001255s" totalImagesPullingTime="1.569116083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:16.475673064 +0000 UTC m=+74.769609866" watchObservedRunningTime="2026-07-29 15:12:16.476001255 +0000 UTC m=+74.769938053"
Jul 29 15:12:17.461246 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:17.461213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5" event={"ID":"96198947-8a41-49ad-baeb-c8d609ef2cb8","Type":"ContainerStarted","Data":"02494d802bbab9ac306fc4ee3f80851032461c8ec6b206c344ce971415c97031"}
Jul 29 15:12:17.483248 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:17.483205    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5" podStartSLOduration=2.295714205 podStartE2EDuration="4.483175736s" totalImagesPullingTime="2.187461531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:17.482264556 +0000 UTC m=+75.776201357" watchObservedRunningTime="2026-07-29 15:12:17.483175736 +0000 UTC m=+75.777112537"
Jul 29 15:12:17.699431 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:17.699387    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:17.699612 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:17.699543    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 15:12:17.699612 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:17.699611    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert podName:6875b039-7cc3-425b-a5aa-c45f0d63d01a nodeName:}" failed. No retries permitted until 2026-07-29 15:12:21.699595294 +0000 UTC m=+79.993532074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rrds9" (UID: "6875b039-7cc3-425b-a5aa-c45f0d63d01a") : secret "networking-console-plugin-cert" not found
Jul 29 15:12:17.800705 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:17.800615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:17.800840 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:17.800800    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:17.800891 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:17.800880    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls podName:d8b85fba-8719-4dee-a1b5-e94d9747836b nodeName:}" failed. No retries permitted until 2026-07-29 15:12:21.800859918 +0000 UTC m=+80.094796702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ccv7r" (UID: "d8b85fba-8719-4dee-a1b5-e94d9747836b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:18.767864 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.767832    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"]
Jul 29 15:12:18.771017 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.771000    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"
Jul 29 15:12:18.773908 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.773879    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 29 15:12:18.774765 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.774740    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-n2mdc\""
Jul 29 15:12:18.774869 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.774742    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 29 15:12:18.780763 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.780739    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"]
Jul 29 15:12:18.808994 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.808965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx8dr\" (UniqueName: \"kubernetes.io/projected/885663bb-5d2f-4c90-b7f8-1004f54cd1a8-kube-api-access-tx8dr\") pod \"migrator-6865c9967-nh6zz\" (UID: \"885663bb-5d2f-4c90-b7f8-1004f54cd1a8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"
Jul 29 15:12:18.909959 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.909906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx8dr\" (UniqueName: \"kubernetes.io/projected/885663bb-5d2f-4c90-b7f8-1004f54cd1a8-kube-api-access-tx8dr\") pod \"migrator-6865c9967-nh6zz\" (UID: \"885663bb-5d2f-4c90-b7f8-1004f54cd1a8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"
Jul 29 15:12:18.920646 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:18.920620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx8dr\" (UniqueName: \"kubernetes.io/projected/885663bb-5d2f-4c90-b7f8-1004f54cd1a8-kube-api-access-tx8dr\") pod \"migrator-6865c9967-nh6zz\" (UID: \"885663bb-5d2f-4c90-b7f8-1004f54cd1a8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"
Jul 29 15:12:19.080012 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:19.079923    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"
Jul 29 15:12:19.202075 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:19.202047    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz"]
Jul 29 15:12:19.205643 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:19.205618    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885663bb_5d2f_4c90_b7f8_1004f54cd1a8.slice/crio-3e7c59fddb5ed62e106ebb16cc9e4b4b6fad675c9a1c9f15df0e44f4abe13ebe WatchSource:0}: Error finding container 3e7c59fddb5ed62e106ebb16cc9e4b4b6fad675c9a1c9f15df0e44f4abe13ebe: Status 404 returned error can't find the container with id 3e7c59fddb5ed62e106ebb16cc9e4b4b6fad675c9a1c9f15df0e44f4abe13ebe
Jul 29 15:12:19.470979 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:19.470944    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz" event={"ID":"885663bb-5d2f-4c90-b7f8-1004f54cd1a8","Type":"ContainerStarted","Data":"3e7c59fddb5ed62e106ebb16cc9e4b4b6fad675c9a1c9f15df0e44f4abe13ebe"}
Jul 29 15:12:21.089718 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.089677    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7p52g"]
Jul 29 15:12:21.093202 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.093164    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.095841 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.095818    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ztgq7\""
Jul 29 15:12:21.096599 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.096570    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 15:12:21.096599 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.096582    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 15:12:21.096789 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.096570    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 15:12:21.096789 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.096571    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 15:12:21.103026 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.102996    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7p52g"]
Jul 29 15:12:21.129436 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.129394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.129436 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.129440    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpc9z\" (UniqueName: \"kubernetes.io/projected/caed7015-d329-411d-891e-69649ef42740-kube-api-access-mpc9z\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.129683 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.129481    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/caed7015-d329-411d-891e-69649ef42740-crio-socket\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.129683 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.129502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.129789 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.129690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/caed7015-d329-411d-891e-69649ef42740-data-volume\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.230933 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.230897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/caed7015-d329-411d-891e-69649ef42740-data-volume\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.231118 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.230948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.231118 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.230972    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpc9z\" (UniqueName: \"kubernetes.io/projected/caed7015-d329-411d-891e-69649ef42740-kube-api-access-mpc9z\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.231118 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.231016    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/caed7015-d329-411d-891e-69649ef42740-crio-socket\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.231118 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.231042    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.231118 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.231093    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.231401 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.231156    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.231401 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.231155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/caed7015-d329-411d-891e-69649ef42740-crio-socket\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.231401 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.231162    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:21.731141753 +0000 UTC m=+80.025078542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.231401 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.231239    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:21.731223783 +0000 UTC m=+80.025160563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.231401 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.231317    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/caed7015-d329-411d-891e-69649ef42740-data-volume\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.243231 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.243204    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpc9z\" (UniqueName: \"kubernetes.io/projected/caed7015-d329-411d-891e-69649ef42740-kube-api-access-mpc9z\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.735416 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.735309    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.735578 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.735426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:21.735578 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.735445    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.735578 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.735451    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:21.735578 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.735520    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.735578 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.735524    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:22.735502153 +0000 UTC m=+81.029438936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.735578 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.735571    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:22.735559982 +0000 UTC m=+81.029496765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.735788 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.735591    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 15:12:21.735788 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.735644    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert podName:6875b039-7cc3-425b-a5aa-c45f0d63d01a nodeName:}" failed. No retries permitted until 2026-07-29 15:12:29.735631166 +0000 UTC m=+88.029567963 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rrds9" (UID: "6875b039-7cc3-425b-a5aa-c45f0d63d01a") : secret "networking-console-plugin-cert" not found
Jul 29 15:12:21.836886 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:21.836841    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:21.837075 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.836992    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:21.837075 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:21.837065    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls podName:d8b85fba-8719-4dee-a1b5-e94d9747836b nodeName:}" failed. No retries permitted until 2026-07-29 15:12:29.837050263 +0000 UTC m=+88.130987044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ccv7r" (UID: "d8b85fba-8719-4dee-a1b5-e94d9747836b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:22.481969 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:22.481922    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz" event={"ID":"885663bb-5d2f-4c90-b7f8-1004f54cd1a8","Type":"ContainerStarted","Data":"54bd6f37957b93b9853685b43f291afbc9713e8821a1063c36447efcad428347"}
Jul 29 15:12:22.482373 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:22.482029    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz" event={"ID":"885663bb-5d2f-4c90-b7f8-1004f54cd1a8","Type":"ContainerStarted","Data":"625502175fb94ce9ec23341a48771e0272d2e29aee081e78bcc53b2c0ec10c79"}
Jul 29 15:12:22.508894 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:22.508838    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nh6zz" podStartSLOduration=2.257433476 podStartE2EDuration="4.508822592s" totalImagesPullingTime="2.251389116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:22.50802016 +0000 UTC m=+80.801956961" watchObservedRunningTime="2026-07-29 15:12:22.508822592 +0000 UTC m=+80.802759394"
Jul 29 15:12:22.744041 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:22.743944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:22.744041 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:22.743995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:22.744315 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:22.744082    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:22.744315 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:22.744123    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:22.744315 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:22.744145    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:24.74412873 +0000 UTC m=+83.038065511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:22.744315 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:22.744221    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:24.74417876 +0000 UTC m=+83.038115561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:23.878662 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:23.878541    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6z2cx_35b71621-959c-4601-9800-f5e3f26182fa/dns-node-resolver/0.log"
Jul 29 15:12:24.681162 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:24.681132    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6b67r_52224984-89d4-44bb-8119-2a95db6c1d03/node-ca/0.log"
Jul 29 15:12:24.758919 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:24.758886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:24.759062 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:24.758945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:24.759062 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:24.759032    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:24.759144 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:24.759073    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:24.759144 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:24.759106    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:28.759090904 +0000 UTC m=+87.053027685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:24.759144 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:24.759121    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:28.759114725 +0000 UTC m=+87.053051505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:25.480446 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:25.480421    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dpl5l_c46543f0-2496-4e4d-9a40-4cc67663aedf/serve-healthcheck-canary/0.log"
Jul 29 15:12:26.079290 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:26.079255    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-nh6zz_885663bb-5d2f-4c90-b7f8-1004f54cd1a8/migrator/0.log"
Jul 29 15:12:26.285465 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:26.285440    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-nh6zz_885663bb-5d2f-4c90-b7f8-1004f54cd1a8/graceful-termination/0.log"
Jul 29 15:12:26.481106 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:26.481071    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5_96198947-8a41-49ad-baeb-c8d609ef2cb8/kube-storage-version-migrator-operator/0.log"
Jul 29 15:12:28.790607 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:28.790568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:28.791013 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:28.790629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:28.791013 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:28.790722    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:28.791013 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:28.790740    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:28.791013 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:28.790788    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:36.790772253 +0000 UTC m=+95.084709033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:28.791013 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:28.790803    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls podName:caed7015-d329-411d-891e-69649ef42740 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:36.790797258 +0000 UTC m=+95.084734038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p52g" (UID: "caed7015-d329-411d-891e-69649ef42740") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:29.799160 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:29.799108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:29.799572 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:29.799286    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 15:12:29.799572 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:29.799355    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert podName:6875b039-7cc3-425b-a5aa-c45f0d63d01a nodeName:}" failed. No retries permitted until 2026-07-29 15:12:45.799339497 +0000 UTC m=+104.093276287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rrds9" (UID: "6875b039-7cc3-425b-a5aa-c45f0d63d01a") : secret "networking-console-plugin-cert" not found
Jul 29 15:12:29.900488 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:29.900459    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:29.900636 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:29.900610    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:29.900686 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:12:29.900675    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls podName:d8b85fba-8719-4dee-a1b5-e94d9747836b nodeName:}" failed. No retries permitted until 2026-07-29 15:12:45.900660447 +0000 UTC m=+104.194597227 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ccv7r" (UID: "d8b85fba-8719-4dee-a1b5-e94d9747836b") : secret "cluster-monitoring-operator-tls" not found
Jul 29 15:12:36.853924 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:36.853880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:36.853924 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:36.853931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:36.854429 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:36.854412    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caed7015-d329-411d-891e-69649ef42740-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:36.856253 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:36.856233    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caed7015-d329-411d-891e-69649ef42740-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p52g\" (UID: \"caed7015-d329-411d-891e-69649ef42740\") " pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:37.006476 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:37.006431    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7p52g"
Jul 29 15:12:37.134450 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:37.134367    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7p52g"]
Jul 29 15:12:37.137484 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:37.137445    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaed7015_d329_411d_891e_69649ef42740.slice/crio-f946e57139ba7831b8ab9995ce1f8d29baa18f9fb9736c0ea56e73ad2558873f WatchSource:0}: Error finding container f946e57139ba7831b8ab9995ce1f8d29baa18f9fb9736c0ea56e73ad2558873f: Status 404 returned error can't find the container with id f946e57139ba7831b8ab9995ce1f8d29baa18f9fb9736c0ea56e73ad2558873f
Jul 29 15:12:37.532524 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:37.532434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p52g" event={"ID":"caed7015-d329-411d-891e-69649ef42740","Type":"ContainerStarted","Data":"f946e57139ba7831b8ab9995ce1f8d29baa18f9fb9736c0ea56e73ad2558873f"}
Jul 29 15:12:39.070673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.070632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:12:39.071142 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.070711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:12:39.073489 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.073453    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdf8ca9f-0611-463b-b647-e7809c88f8c1-metrics-tls\") pod \"dns-default-h79ts\" (UID: \"bdf8ca9f-0611-463b-b647-e7809c88f8c1\") " pod="openshift-dns/dns-default-h79ts"
Jul 29 15:12:39.073916 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.073893    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"image-registry-d6d695b6d-9gwgt\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") " pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:12:39.096475 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.096440    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw7hg\""
Jul 29 15:12:39.104553 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.104529    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h79ts"
Jul 29 15:12:39.256745 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.256716    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h79ts"]
Jul 29 15:12:39.259811 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:39.259780    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf8ca9f_0611_463b_b647_e7809c88f8c1.slice/crio-5a156df53b29adabd2a828c45928ff28b73405a5fc1d732d29b7879c3c6ef1ec WatchSource:0}: Error finding container 5a156df53b29adabd2a828c45928ff28b73405a5fc1d732d29b7879c3c6ef1ec: Status 404 returned error can't find the container with id 5a156df53b29adabd2a828c45928ff28b73405a5fc1d732d29b7879c3c6ef1ec
Jul 29 15:12:39.363583 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.363551    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-snrjc\""
Jul 29 15:12:39.371470 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.371441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:12:39.534536 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.534500    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d6d695b6d-9gwgt"]
Jul 29 15:12:39.539811 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:39.539759    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf77d19c_fa4d_41aa_9d0b_3ed1a7c49bab.slice/crio-e864e4821946a8f84e9a1e672b30869046e46715e3fb0fa9acfe7208cb93806a WatchSource:0}: Error finding container e864e4821946a8f84e9a1e672b30869046e46715e3fb0fa9acfe7208cb93806a: Status 404 returned error can't find the container with id e864e4821946a8f84e9a1e672b30869046e46715e3fb0fa9acfe7208cb93806a
Jul 29 15:12:39.540646 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:39.540617    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h79ts" event={"ID":"bdf8ca9f-0611-463b-b647-e7809c88f8c1","Type":"ContainerStarted","Data":"5a156df53b29adabd2a828c45928ff28b73405a5fc1d732d29b7879c3c6ef1ec"}
Jul 29 15:12:40.545377 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:40.545340    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt" event={"ID":"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab","Type":"ContainerStarted","Data":"b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733"}
Jul 29 15:12:40.545777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:40.545401    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt" event={"ID":"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab","Type":"ContainerStarted","Data":"e864e4821946a8f84e9a1e672b30869046e46715e3fb0fa9acfe7208cb93806a"}
Jul 29 15:12:40.545777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:40.545489    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:12:41.435758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:41.435727    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b7zwb"
Jul 29 15:12:41.459766 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:41.459712    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt" podStartSLOduration=98.459689172 podStartE2EDuration="1m38.459689172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:40.567824877 +0000 UTC m=+98.861761692" watchObservedRunningTime="2026-07-29 15:12:41.459689172 +0000 UTC m=+99.753625975"
Jul 29 15:12:43.556892 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:43.556857    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p52g" event={"ID":"caed7015-d329-411d-891e-69649ef42740","Type":"ContainerStarted","Data":"ff13226bd1ac291494ec4d1b7cc4fa6eec20d1a011adc8d9049c250eee143a37"}
Jul 29 15:12:43.558398 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:43.558376    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h79ts" event={"ID":"bdf8ca9f-0611-463b-b647-e7809c88f8c1","Type":"ContainerStarted","Data":"9e4c3cd0922e8458b23c9d4be42c5b49876f20a50aa7fa76d589bdc33ed5fb04"}
Jul 29 15:12:43.558539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:43.558413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h79ts" event={"ID":"bdf8ca9f-0611-463b-b647-e7809c88f8c1","Type":"ContainerStarted","Data":"e2adef347df11d3e0f1eb4aa133095f7f5f08c1307b486fcd7d5c74add4e3abe"}
Jul 29 15:12:43.558539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:43.558506    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h79ts"
Jul 29 15:12:43.581677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:43.581628    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h79ts" podStartSLOduration=64.824351662 podStartE2EDuration="1m8.581609699s" totalImagesPullingTime="3.757258037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:43.579956473 +0000 UTC m=+101.873893271" watchObservedRunningTime="2026-07-29 15:12:43.581609699 +0000 UTC m=+101.875546503"
Jul 29 15:12:44.358328 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.358279    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"]
Jul 29 15:12:44.370681 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.370649    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:44.372912 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.372882    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"]
Jul 29 15:12:44.374465 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.374401    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jhxz4\""
Jul 29 15:12:44.374611 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.374526    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 15:12:44.374611 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.374567    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 15:12:44.374725 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.374664    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 15:12:44.374852 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.374795    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 15:12:44.515938 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.515904    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg886\" (UniqueName: \"kubernetes.io/projected/a6f852eb-74cf-4621-8883-e8181d56ee2f-kube-api-access-cg886\") pod \"managed-serviceaccount-addon-agent-8496fd5586-g5ps6\" (UID: \"a6f852eb-74cf-4621-8883-e8181d56ee2f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:44.515938 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.515948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a6f852eb-74cf-4621-8883-e8181d56ee2f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8496fd5586-g5ps6\" (UID: \"a6f852eb-74cf-4621-8883-e8181d56ee2f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:44.616708 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.616622    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg886\" (UniqueName: \"kubernetes.io/projected/a6f852eb-74cf-4621-8883-e8181d56ee2f-kube-api-access-cg886\") pod \"managed-serviceaccount-addon-agent-8496fd5586-g5ps6\" (UID: \"a6f852eb-74cf-4621-8883-e8181d56ee2f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:44.616708 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.616661    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a6f852eb-74cf-4621-8883-e8181d56ee2f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8496fd5586-g5ps6\" (UID: \"a6f852eb-74cf-4621-8883-e8181d56ee2f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:44.619155 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.619131    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a6f852eb-74cf-4621-8883-e8181d56ee2f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8496fd5586-g5ps6\" (UID: \"a6f852eb-74cf-4621-8883-e8181d56ee2f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:44.626967 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.626940    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg886\" (UniqueName: \"kubernetes.io/projected/a6f852eb-74cf-4621-8883-e8181d56ee2f-kube-api-access-cg886\") pod \"managed-serviceaccount-addon-agent-8496fd5586-g5ps6\" (UID: \"a6f852eb-74cf-4621-8883-e8181d56ee2f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:44.696387 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:44.696345    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"
Jul 29 15:12:45.827761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:45.827714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:45.830044 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:45.830023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6875b039-7cc3-425b-a5aa-c45f0d63d01a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rrds9\" (UID: \"6875b039-7cc3-425b-a5aa-c45f0d63d01a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:45.928361 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:45.928323    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:45.930662 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:45.930638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8b85fba-8719-4dee-a1b5-e94d9747836b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ccv7r\" (UID: \"d8b85fba-8719-4dee-a1b5-e94d9747836b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:45.930796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:45.930732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"
Jul 29 15:12:46.117327 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:46.117268    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"
Jul 29 15:12:47.217985 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:47.217949    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rrds9"]
Jul 29 15:12:47.220858 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:47.220833    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6875b039_7cc3_425b_a5aa_c45f0d63d01a.slice/crio-e863d109d1679e048c7890741a6ae75fed3c100e690be0343ad9da519b018617 WatchSource:0}: Error finding container e863d109d1679e048c7890741a6ae75fed3c100e690be0343ad9da519b018617: Status 404 returned error can't find the container with id e863d109d1679e048c7890741a6ae75fed3c100e690be0343ad9da519b018617
Jul 29 15:12:47.231623 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:47.231599    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6"]
Jul 29 15:12:47.250686 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:47.250657    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r"]
Jul 29 15:12:47.411236 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:47.411198    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6f852eb_74cf_4621_8883_e8181d56ee2f.slice/crio-f000e9c56aff6a3f59abae81577b5bb5d517664facdc37512e173c6f604092ce WatchSource:0}: Error finding container f000e9c56aff6a3f59abae81577b5bb5d517664facdc37512e173c6f604092ce: Status 404 returned error can't find the container with id f000e9c56aff6a3f59abae81577b5bb5d517664facdc37512e173c6f604092ce
Jul 29 15:12:47.411609 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:47.411583    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b85fba_8719_4dee_a1b5_e94d9747836b.slice/crio-e23ee4e40296df30948fce10aeddc3be001e657d37c973b6f2fa8d6d0869e061 WatchSource:0}: Error finding container e23ee4e40296df30948fce10aeddc3be001e657d37c973b6f2fa8d6d0869e061: Status 404 returned error can't find the container with id e23ee4e40296df30948fce10aeddc3be001e657d37c973b6f2fa8d6d0869e061
Jul 29 15:12:47.573955 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:47.573920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9" event={"ID":"6875b039-7cc3-425b-a5aa-c45f0d63d01a","Type":"ContainerStarted","Data":"e863d109d1679e048c7890741a6ae75fed3c100e690be0343ad9da519b018617"}
Jul 29 15:12:47.574960 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:47.574937    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6" event={"ID":"a6f852eb-74cf-4621-8883-e8181d56ee2f","Type":"ContainerStarted","Data":"f000e9c56aff6a3f59abae81577b5bb5d517664facdc37512e173c6f604092ce"}
Jul 29 15:12:47.576063 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:47.576044    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r" event={"ID":"d8b85fba-8719-4dee-a1b5-e94d9747836b","Type":"ContainerStarted","Data":"e23ee4e40296df30948fce10aeddc3be001e657d37c973b6f2fa8d6d0869e061"}
Jul 29 15:12:47.577572 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:47.577553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p52g" event={"ID":"caed7015-d329-411d-891e-69649ef42740","Type":"ContainerStarted","Data":"889f0176086f1da47316042c7a405ac44bf666fccc7fc6b9fefd11b4ebffe7c7"}
Jul 29 15:12:52.607958 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.607865    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9" event={"ID":"6875b039-7cc3-425b-a5aa-c45f0d63d01a","Type":"ContainerStarted","Data":"8a798d498e8af10935a3244ef465e207e6a177a025c6a72d49101a44636aeab3"}
Jul 29 15:12:52.609634 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.609603    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6" event={"ID":"a6f852eb-74cf-4621-8883-e8181d56ee2f","Type":"ContainerStarted","Data":"d63a98ef05b7ce31f32aa12a1f5e02ca3964ed88a29c230b58ff0c22ed20965e"}
Jul 29 15:12:52.611054 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.611034    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r" event={"ID":"d8b85fba-8719-4dee-a1b5-e94d9747836b","Type":"ContainerStarted","Data":"b12fa0d532d97eb9a2c3d179146e8d75aae8314fe002bc035fb76ace626b5041"}
Jul 29 15:12:52.613050 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.613028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p52g" event={"ID":"caed7015-d329-411d-891e-69649ef42740","Type":"ContainerStarted","Data":"a6e99515625da0bfcee94e2d64828533985cd561d23c21b005c6e992bab3ffe3"}
Jul 29 15:12:52.652711 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.652654    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rrds9" podStartSLOduration=34.521685548 podStartE2EDuration="39.652638703s" totalImagesPullingTime="5.130953155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:52.651401755 +0000 UTC m=+110.945338548" watchObservedRunningTime="2026-07-29 15:12:52.652638703 +0000 UTC m=+110.946575504"
Jul 29 15:12:52.671469 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.671417    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r" podStartSLOduration=33.749626366 podStartE2EDuration="38.671397335s" totalImagesPullingTime="4.921770969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:52.670458871 +0000 UTC m=+110.964395674" watchObservedRunningTime="2026-07-29 15:12:52.671397335 +0000 UTC m=+110.965334139"
Jul 29 15:12:52.700692 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.700639    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7p52g" podStartSLOduration=16.622474271 podStartE2EDuration="31.700620493s" totalImagesPullingTime="15.078146222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:12:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:52.699320134 +0000 UTC m=+110.993256931" watchObservedRunningTime="2026-07-29 15:12:52.700620493 +0000 UTC m=+110.994557295"
Jul 29 15:12:52.722015 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.721963    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8496fd5586-g5ps6" podStartSLOduration=3.741148477 podStartE2EDuration="8.721947412s" totalImagesPullingTime="4.980798935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:52.721385793 +0000 UTC m=+111.015322596" watchObservedRunningTime="2026-07-29 15:12:52.721947412 +0000 UTC m=+111.015884213"
Jul 29 15:12:52.945341 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.945258    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"]
Jul 29 15:12:52.948694 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.948674    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"
Jul 29 15:12:52.951962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.951938    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 15:12:52.952102 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.952015    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5v6wp\""
Jul 29 15:12:52.962394 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:52.962371    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"]
Jul 29 15:12:53.093736 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:53.093696    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/69316d5f-c98d-43a6-95ed-de7442c232dd-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-mmdqn\" (UID: \"69316d5f-c98d-43a6-95ed-de7442c232dd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"
Jul 29 15:12:53.194473 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:53.194431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/69316d5f-c98d-43a6-95ed-de7442c232dd-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-mmdqn\" (UID: \"69316d5f-c98d-43a6-95ed-de7442c232dd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"
Jul 29 15:12:53.196904 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:53.196837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/69316d5f-c98d-43a6-95ed-de7442c232dd-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-mmdqn\" (UID: \"69316d5f-c98d-43a6-95ed-de7442c232dd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"
Jul 29 15:12:53.258633 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:53.258598    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"
Jul 29 15:12:53.391418 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:53.391386    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"]
Jul 29 15:12:53.394670 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:12:53.394640    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69316d5f_c98d_43a6_95ed_de7442c232dd.slice/crio-b1de5a3badd604d790669a29029410c0b09f088cb1d06c5cd8ca527de0533366 WatchSource:0}: Error finding container b1de5a3badd604d790669a29029410c0b09f088cb1d06c5cd8ca527de0533366: Status 404 returned error can't find the container with id b1de5a3badd604d790669a29029410c0b09f088cb1d06c5cd8ca527de0533366
Jul 29 15:12:53.563666 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:53.563588    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h79ts"
Jul 29 15:12:53.617922 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:53.617888    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn" event={"ID":"69316d5f-c98d-43a6-95ed-de7442c232dd","Type":"ContainerStarted","Data":"b1de5a3badd604d790669a29029410c0b09f088cb1d06c5cd8ca527de0533366"}
Jul 29 15:12:55.626291 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:55.626259    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn" event={"ID":"69316d5f-c98d-43a6-95ed-de7442c232dd","Type":"ContainerStarted","Data":"606cf875ae9597c01da1f6205ecaae2e52375ebc10adb72327d82049644e1d83"}
Jul 29 15:12:55.626769 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:55.626399    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"
Jul 29 15:12:55.630935 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:55.630912    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn"
Jul 29 15:12:55.645577 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:55.645504    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-mmdqn" podStartSLOduration=2.429229689 podStartE2EDuration="3.645490667s" totalImagesPullingTime="1.216260978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:55.644524618 +0000 UTC m=+113.938461419" watchObservedRunningTime="2026-07-29 15:12:55.645490667 +0000 UTC m=+113.939427469"
Jul 29 15:12:59.376411 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:59.376375    2560 patch_prober.go:28] interesting pod/image-registry-d6d695b6d-9gwgt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 29 15:12:59.376807 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:12:59.376430    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt" podUID="bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 15:13:00.644347 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:00.644321    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:13:00.644717 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:00.644357    2560 generic.go:356] "Generic (PLEG): container finished" podID="d8b85fba-8719-4dee-a1b5-e94d9747836b" containerID="b12fa0d532d97eb9a2c3d179146e8d75aae8314fe002bc035fb76ace626b5041" exitCode=2
Jul 29 15:13:00.644717 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:00.644383    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r" event={"ID":"d8b85fba-8719-4dee-a1b5-e94d9747836b","Type":"ContainerDied","Data":"b12fa0d532d97eb9a2c3d179146e8d75aae8314fe002bc035fb76ace626b5041"}
Jul 29 15:13:00.644717 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:00.644657    2560 scope.go:117] "RemoveContainer" containerID="b12fa0d532d97eb9a2c3d179146e8d75aae8314fe002bc035fb76ace626b5041"
Jul 29 15:13:01.383325 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.383291    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-dwq45"]
Jul 29 15:13:01.386391 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.386373    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-dwq45"
Jul 29 15:13:01.388947 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.388923    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-nv82x\""
Jul 29 15:13:01.389074 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.389002    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 15:13:01.389136 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.389115    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 15:13:01.398113 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.398090    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-dwq45"]
Jul 29 15:13:01.458117 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.458085    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx845\" (UniqueName: \"kubernetes.io/projected/7929f79e-0591-475b-aa42-17f4bc2b24d2-kube-api-access-cx845\") pod \"downloads-7c97dc4555-dwq45\" (UID: \"7929f79e-0591-475b-aa42-17f4bc2b24d2\") " pod="openshift-console/downloads-7c97dc4555-dwq45"
Jul 29 15:13:01.553681 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.553654    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:13:01.559014 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.558984    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx845\" (UniqueName: \"kubernetes.io/projected/7929f79e-0591-475b-aa42-17f4bc2b24d2-kube-api-access-cx845\") pod \"downloads-7c97dc4555-dwq45\" (UID: \"7929f79e-0591-475b-aa42-17f4bc2b24d2\") " pod="openshift-console/downloads-7c97dc4555-dwq45"
Jul 29 15:13:01.570833 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.570806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx845\" (UniqueName: \"kubernetes.io/projected/7929f79e-0591-475b-aa42-17f4bc2b24d2-kube-api-access-cx845\") pod \"downloads-7c97dc4555-dwq45\" (UID: \"7929f79e-0591-475b-aa42-17f4bc2b24d2\") " pod="openshift-console/downloads-7c97dc4555-dwq45"
Jul 29 15:13:01.651236 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.651141    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:13:01.651590 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.651252    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ccv7r" event={"ID":"d8b85fba-8719-4dee-a1b5-e94d9747836b","Type":"ContainerStarted","Data":"8718e61cfa3f7ebf54ab9f334d2606e234322ec16fcacb051a0e65eabcaa55df"}
Jul 29 15:13:01.696779 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.696747    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-dwq45"
Jul 29 15:13:01.830433 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:01.830407    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-dwq45"]
Jul 29 15:13:01.832908 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:13:01.832877    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7929f79e_0591_475b_aa42_17f4bc2b24d2.slice/crio-b077886347faee7a883d373a55c9012d094c89dee9f23a9c11406fa9e5f8d0af WatchSource:0}: Error finding container b077886347faee7a883d373a55c9012d094c89dee9f23a9c11406fa9e5f8d0af: Status 404 returned error can't find the container with id b077886347faee7a883d373a55c9012d094c89dee9f23a9c11406fa9e5f8d0af
Jul 29 15:13:02.657134 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:02.657092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-dwq45" event={"ID":"7929f79e-0591-475b-aa42-17f4bc2b24d2","Type":"ContainerStarted","Data":"b077886347faee7a883d373a55c9012d094c89dee9f23a9c11406fa9e5f8d0af"}
Jul 29 15:13:04.333691 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.333655    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"]
Jul 29 15:13:04.337085 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.337056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.339602 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.339553    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bqmv4\""
Jul 29 15:13:04.340290 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.340267    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 15:13:04.340417 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.340314    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 15:13:04.340417 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.340331    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 15:13:04.354055 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.354025    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"]
Jul 29 15:13:04.380338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.380309    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2027a2a-60f4-443d-8ef2-e2917c68c983-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.380510 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.380353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whzg2\" (UniqueName: \"kubernetes.io/projected/c2027a2a-60f4-443d-8ef2-e2917c68c983-kube-api-access-whzg2\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.380510 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.380377    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2027a2a-60f4-443d-8ef2-e2917c68c983-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.380510 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.380481    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2027a2a-60f4-443d-8ef2-e2917c68c983-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.470593 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.470561    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"]
Jul 29 15:13:04.474323 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.474300    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.476813 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.476788    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 15:13:04.477310 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.477291    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 15:13:04.477697 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.477639    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 15:13:04.477788 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.477718    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j8xfj\""
Jul 29 15:13:04.481121 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/89eff43c-1c38-408c-8338-2357ac807bbf-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.481487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.481595 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5lkm\" (UniqueName: \"kubernetes.io/projected/89eff43c-1c38-408c-8338-2357ac807bbf-kube-api-access-g5lkm\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.481595 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481521    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.481595 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481562    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.481758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481616    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2027a2a-60f4-443d-8ef2-e2917c68c983-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.481758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2027a2a-60f4-443d-8ef2-e2917c68c983-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.481758 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481708    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whzg2\" (UniqueName: \"kubernetes.io/projected/c2027a2a-60f4-443d-8ef2-e2917c68c983-kube-api-access-whzg2\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.481956 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2027a2a-60f4-443d-8ef2-e2917c68c983-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.481956 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.481822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89eff43c-1c38-408c-8338-2357ac807bbf-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.482668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.482646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2027a2a-60f4-443d-8ef2-e2917c68c983-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.484855 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.484834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2027a2a-60f4-443d-8ef2-e2917c68c983-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.484955 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.484837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2027a2a-60f4-443d-8ef2-e2917c68c983-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.501244 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.501211    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"]
Jul 29 15:13:04.506695 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.506630    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whzg2\" (UniqueName: \"kubernetes.io/projected/c2027a2a-60f4-443d-8ef2-e2917c68c983-kube-api-access-whzg2\") pod \"openshift-state-metrics-b8f49f5c6-gcd5g\" (UID: \"c2027a2a-60f4-443d-8ef2-e2917c68c983\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.508609 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.508562    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s2ddf"]
Jul 29 15:13:04.512650 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.512630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.515608 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.515577    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rlsh5\""
Jul 29 15:13:04.515608 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.515593    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 15:13:04.515857 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.515837    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 15:13:04.518279 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.518257    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 15:13:04.583133 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583097    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-textfile\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.583355 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583149    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/89eff43c-1c38-408c-8338-2357ac807bbf-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.583355 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583176    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.583677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583645    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/89eff43c-1c38-408c-8338-2357ac807bbf-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.583806 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmmj5\" (UniqueName: \"kubernetes.io/projected/5483010b-120e-40b4-874b-ce9e21436739-kube-api-access-tmmj5\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.583806 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583762    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5lkm\" (UniqueName: \"kubernetes.io/projected/89eff43c-1c38-408c-8338-2357ac807bbf-kube-api-access-g5lkm\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.583931 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-sys\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.583931 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583851    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.583931 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.584074 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.583967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-root\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.584074 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.584011    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.584074 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.584030    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.584262 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:13:04.584077    2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 15:13:04.584262 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:13:04.584145    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-tls podName:89eff43c-1c38-408c-8338-2357ac807bbf nodeName:}" failed. No retries permitted until 2026-07-29 15:13:05.084126819 +0000 UTC m=+123.378063599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-ztszl" (UID: "89eff43c-1c38-408c-8338-2357ac807bbf") : secret "kube-state-metrics-tls" not found
Jul 29 15:13:04.584376 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.584077    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-tls\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.584376 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.584325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5483010b-120e-40b4-874b-ce9e21436739-metrics-client-ca\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.584376 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.584357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-accelerators-collector-config\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.584522 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.584394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-wtmp\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.584522 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.584431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89eff43c-1c38-408c-8338-2357ac807bbf-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.585029 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.585005    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89eff43c-1c38-408c-8338-2357ac807bbf-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.586776 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.586753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.595223 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.595202    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5lkm\" (UniqueName: \"kubernetes.io/projected/89eff43c-1c38-408c-8338-2357ac807bbf-kube-api-access-g5lkm\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:04.649587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.649549    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"
Jul 29 15:13:04.685819 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.685773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-root\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.685986 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.685837    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.685986 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.685866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-tls\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.685986 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.685870    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-root\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.685986 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.685890    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5483010b-120e-40b4-874b-ce9e21436739-metrics-client-ca\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.685986 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.685918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-accelerators-collector-config\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.685986 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.685954    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-wtmp\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686315 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686129    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-wtmp\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686315 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-textfile\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686315 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmmj5\" (UniqueName: \"kubernetes.io/projected/5483010b-120e-40b4-874b-ce9e21436739-kube-api-access-tmmj5\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686461 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686336    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-sys\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686461 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686424    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5483010b-120e-40b4-874b-ce9e21436739-sys\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686627 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686606    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-textfile\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686760 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686682    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5483010b-120e-40b4-874b-ce9e21436739-metrics-client-ca\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.686760 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.686698    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-accelerators-collector-config\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.689070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.689049    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.689160 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.689078    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5483010b-120e-40b4-874b-ce9e21436739-node-exporter-tls\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.698992 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.698960    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmmj5\" (UniqueName: \"kubernetes.io/projected/5483010b-120e-40b4-874b-ce9e21436739-kube-api-access-tmmj5\") pod \"node-exporter-s2ddf\" (UID: \"5483010b-120e-40b4-874b-ce9e21436739\") " pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.800953 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.800815    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g"]
Jul 29 15:13:04.803310 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:13:04.803279    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2027a2a_60f4_443d_8ef2_e2917c68c983.slice/crio-d5ad2f611f673eeafe5aaff3c8cc410f767f18a0939780857c74a3621682c505 WatchSource:0}: Error finding container d5ad2f611f673eeafe5aaff3c8cc410f767f18a0939780857c74a3621682c505: Status 404 returned error can't find the container with id d5ad2f611f673eeafe5aaff3c8cc410f767f18a0939780857c74a3621682c505
Jul 29 15:13:04.824080 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:04.824056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s2ddf"
Jul 29 15:13:04.835741 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:13:04.835711    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5483010b_120e_40b4_874b_ce9e21436739.slice/crio-5e6e0e4681450a8c25bf4f273b51d207496168959b3b59ac00341d247f199a00 WatchSource:0}: Error finding container 5e6e0e4681450a8c25bf4f273b51d207496168959b3b59ac00341d247f199a00: Status 404 returned error can't find the container with id 5e6e0e4681450a8c25bf4f273b51d207496168959b3b59ac00341d247f199a00
Jul 29 15:13:05.090050 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.089965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:05.092377 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.092358    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/89eff43c-1c38-408c-8338-2357ac807bbf-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ztszl\" (UID: \"89eff43c-1c38-408c-8338-2357ac807bbf\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:05.386445 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.386407    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"
Jul 29 15:13:05.555227 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.555174    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl"]
Jul 29 15:13:05.671210 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.671108    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g" event={"ID":"c2027a2a-60f4-443d-8ef2-e2917c68c983","Type":"ContainerStarted","Data":"82053d09e7c14da2b7a3ed3faa3776567deae49de7af8e6bf46853541c256a7c"}
Jul 29 15:13:05.671210 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.671175    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g" event={"ID":"c2027a2a-60f4-443d-8ef2-e2917c68c983","Type":"ContainerStarted","Data":"01f3adb41dfcdbc1d54df6b792ba7b1e8e25d32932d2085710df4cbdc395e282"}
Jul 29 15:13:05.671418 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.671227    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g" event={"ID":"c2027a2a-60f4-443d-8ef2-e2917c68c983","Type":"ContainerStarted","Data":"d5ad2f611f673eeafe5aaff3c8cc410f767f18a0939780857c74a3621682c505"}
Jul 29 15:13:05.672640 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:05.672606    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2ddf" event={"ID":"5483010b-120e-40b4-874b-ce9e21436739","Type":"ContainerStarted","Data":"5e6e0e4681450a8c25bf4f273b51d207496168959b3b59ac00341d247f199a00"}
Jul 29 15:13:05.716663 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:13:05.716620    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89eff43c_1c38_408c_8338_2357ac807bbf.slice/crio-ee9a7bb99e47ff2c3fdcddd4b0b737ad61699605f9e28722bd048fb22105370c WatchSource:0}: Error finding container ee9a7bb99e47ff2c3fdcddd4b0b737ad61699605f9e28722bd048fb22105370c: Status 404 returned error can't find the container with id ee9a7bb99e47ff2c3fdcddd4b0b737ad61699605f9e28722bd048fb22105370c
Jul 29 15:13:06.679711 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:06.679616    2560 generic.go:356] "Generic (PLEG): container finished" podID="5483010b-120e-40b4-874b-ce9e21436739" containerID="1ef82eb77f5c016a81b2deb69319d0ccbd5375674d7f87d15fb240f4dacec386" exitCode=0
Jul 29 15:13:06.679711 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:06.679698    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2ddf" event={"ID":"5483010b-120e-40b4-874b-ce9e21436739","Type":"ContainerDied","Data":"1ef82eb77f5c016a81b2deb69319d0ccbd5375674d7f87d15fb240f4dacec386"}
Jul 29 15:13:06.681130 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:06.681073    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl" event={"ID":"89eff43c-1c38-408c-8338-2357ac807bbf","Type":"ContainerStarted","Data":"ee9a7bb99e47ff2c3fdcddd4b0b737ad61699605f9e28722bd048fb22105370c"}
Jul 29 15:13:06.985795 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:06.985429    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d6d695b6d-9gwgt"]
Jul 29 15:13:07.690448 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:07.689740    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2ddf" event={"ID":"5483010b-120e-40b4-874b-ce9e21436739","Type":"ContainerStarted","Data":"5ad6a664ebcc7b172205a4858a0a4fe657aabfe58ce3d1d563dec1b18cf1b317"}
Jul 29 15:13:07.690448 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:07.689857    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2ddf" event={"ID":"5483010b-120e-40b4-874b-ce9e21436739","Type":"ContainerStarted","Data":"5b7580af4eb329ff20b51ffc4dd8932defbb6bf500c639b65747c3c095b179a6"}
Jul 29 15:13:07.692927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:07.692775    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl" event={"ID":"89eff43c-1c38-408c-8338-2357ac807bbf","Type":"ContainerStarted","Data":"3e124259220af3b73444617b34167093bd7cfb3df4efe6c852555f351e9f1476"}
Jul 29 15:13:07.692927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:07.692842    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl" event={"ID":"89eff43c-1c38-408c-8338-2357ac807bbf","Type":"ContainerStarted","Data":"9d612764831cec49f391ab03ea17e5a12963adb2c8082ad02a605863db393713"}
Jul 29 15:13:07.692927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:07.692876    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl" event={"ID":"89eff43c-1c38-408c-8338-2357ac807bbf","Type":"ContainerStarted","Data":"c38038ca4f099c1b8486f2bb77afab1018017fa4866caa038ab7a8ea2f5a1563"}
Jul 29 15:13:07.711927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:07.711804    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s2ddf" podStartSLOduration=2.705277527 podStartE2EDuration="3.711787808s" totalImagesPullingTime="935.438534ms" totalInitContainerRuntime="71.071747ms" isStatefulPod=true podCreationTimestamp="2026-07-29 15:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:07.710686442 +0000 UTC m=+126.004623244" watchObservedRunningTime="2026-07-29 15:13:07.711787808 +0000 UTC m=+126.005724612"
Jul 29 15:13:07.737735 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:07.737681    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ztszl" podStartSLOduration=2.348021487 podStartE2EDuration="3.737665316s" totalImagesPullingTime="1.389643829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:07.73680256 +0000 UTC m=+126.030739363" watchObservedRunningTime="2026-07-29 15:13:07.737665316 +0000 UTC m=+126.031602117"
Jul 29 15:13:08.700543 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.700499    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g" event={"ID":"c2027a2a-60f4-443d-8ef2-e2917c68c983","Type":"ContainerStarted","Data":"e5fb9cadb7cf649614301b9017c048579a7e885316e77850c021c9de87beda27"}
Jul 29 15:13:08.720985 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.720926    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-gcd5g" podStartSLOduration=1.4700178830000001 podStartE2EDuration="4.72090911s" totalImagesPullingTime="3.250891227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:08.719830903 +0000 UTC m=+127.013767705" watchObservedRunningTime="2026-07-29 15:13:08.72090911 +0000 UTC m=+127.014845912"
Jul 29 15:13:08.736270 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.736229    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-67bf84cdff-rkndn"]
Jul 29 15:13:08.739377 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.739358    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.742033 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.742003    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 15:13:08.742033 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.742035    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mfdqq\""
Jul 29 15:13:08.742241 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.742060    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ff6ikdlfgsdpg\""
Jul 29 15:13:08.742241 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.742131    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 15:13:08.742328 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.742258    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 15:13:08.742364 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.742332    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 15:13:08.751552 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.751531    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67bf84cdff-rkndn"]
Jul 29 15:13:08.824260 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.824214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d031dd99-2750-4002-be09-c09655f5f6bf-audit-log\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.824260 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.824259    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-client-ca-bundle\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.824507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.824334    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d031dd99-2750-4002-be09-c09655f5f6bf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.824507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.824374    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d031dd99-2750-4002-be09-c09655f5f6bf-metrics-server-audit-profiles\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.824507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.824397    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-secret-metrics-server-tls\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.824507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.824454    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-secret-metrics-server-client-certs\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.824507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.824478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flq6r\" (UniqueName: \"kubernetes.io/projected/d031dd99-2750-4002-be09-c09655f5f6bf-kube-api-access-flq6r\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925341 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-secret-metrics-server-client-certs\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925341 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flq6r\" (UniqueName: \"kubernetes.io/projected/d031dd99-2750-4002-be09-c09655f5f6bf-kube-api-access-flq6r\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925381    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d031dd99-2750-4002-be09-c09655f5f6bf-audit-log\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925397    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-client-ca-bundle\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925422    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d031dd99-2750-4002-be09-c09655f5f6bf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925442    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d031dd99-2750-4002-be09-c09655f5f6bf-metrics-server-audit-profiles\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-secret-metrics-server-tls\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.925893 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.925861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d031dd99-2750-4002-be09-c09655f5f6bf-audit-log\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.926421 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.926397    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d031dd99-2750-4002-be09-c09655f5f6bf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.926724 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.926702    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d031dd99-2750-4002-be09-c09655f5f6bf-metrics-server-audit-profiles\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.927858 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.927838    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-client-ca-bundle\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.927965 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.927947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-secret-metrics-server-tls\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.928197 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.928162    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d031dd99-2750-4002-be09-c09655f5f6bf-secret-metrics-server-client-certs\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:08.947876 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:08.947852    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flq6r\" (UniqueName: \"kubernetes.io/projected/d031dd99-2750-4002-be09-c09655f5f6bf-kube-api-access-flq6r\") pod \"metrics-server-67bf84cdff-rkndn\" (UID: \"d031dd99-2750-4002-be09-c09655f5f6bf\") " pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:09.049107 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:09.049019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:09.206856 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:09.206827    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67bf84cdff-rkndn"]
Jul 29 15:13:09.210586 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:13:09.210553    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd031dd99_2750_4002_be09_c09655f5f6bf.slice/crio-21465f32cfbf8a3ce20241d395a176562158f1423f57e301add996ee7af4b4f4 WatchSource:0}: Error finding container 21465f32cfbf8a3ce20241d395a176562158f1423f57e301add996ee7af4b4f4: Status 404 returned error can't find the container with id 21465f32cfbf8a3ce20241d395a176562158f1423f57e301add996ee7af4b4f4
Jul 29 15:13:09.705489 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:09.705457    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn" event={"ID":"d031dd99-2750-4002-be09-c09655f5f6bf","Type":"ContainerStarted","Data":"21465f32cfbf8a3ce20241d395a176562158f1423f57e301add996ee7af4b4f4"}
Jul 29 15:13:10.843932 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:10.843890    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:13:10.846616 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:10.846551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2415e25c-6267-4eb0-bb24-11872c713139-metrics-certs\") pod \"network-metrics-daemon-gnjvk\" (UID: \"2415e25c-6267-4eb0-bb24-11872c713139\") " pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:13:10.986988 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:10.986948    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sgb7s\""
Jul 29 15:13:10.995263 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:10.995229    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnjvk"
Jul 29 15:13:11.161125 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:11.161060    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gnjvk"]
Jul 29 15:13:11.165404 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:13:11.165377    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2415e25c_6267_4eb0_bb24_11872c713139.slice/crio-cdf6d53772e69ced0ab8e03cad79eee177c63eefa8ce1290f19e0d8f060a93c9 WatchSource:0}: Error finding container cdf6d53772e69ced0ab8e03cad79eee177c63eefa8ce1290f19e0d8f060a93c9: Status 404 returned error can't find the container with id cdf6d53772e69ced0ab8e03cad79eee177c63eefa8ce1290f19e0d8f060a93c9
Jul 29 15:13:11.715834 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:11.715797    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gnjvk" event={"ID":"2415e25c-6267-4eb0-bb24-11872c713139","Type":"ContainerStarted","Data":"cdf6d53772e69ced0ab8e03cad79eee177c63eefa8ce1290f19e0d8f060a93c9"}
Jul 29 15:13:11.717499 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:11.717464    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn" event={"ID":"d031dd99-2750-4002-be09-c09655f5f6bf","Type":"ContainerStarted","Data":"1132cbbb2e3c4fd14bd9077d8b6784b2a6c16a4fc4b43a4e52ad6779521056ef"}
Jul 29 15:13:11.741102 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:11.741040    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn" podStartSLOduration=2.148239394 podStartE2EDuration="3.74102363s" totalImagesPullingTime="1.592784236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:11.738752351 +0000 UTC m=+130.032689180" watchObservedRunningTime="2026-07-29 15:13:11.74102363 +0000 UTC m=+130.034960431"
Jul 29 15:13:12.725574 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:12.725530    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gnjvk" event={"ID":"2415e25c-6267-4eb0-bb24-11872c713139","Type":"ContainerStarted","Data":"8e684ce4b3fd550743c8e995bd35e17392565099738e0eb6cf8b950c2e34d8e2"}
Jul 29 15:13:12.725981 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:12.725651    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gnjvk" event={"ID":"2415e25c-6267-4eb0-bb24-11872c713139","Type":"ContainerStarted","Data":"e13178cf2c9476ab7f8411372c4c4a55b3c0e54ab77e3c23b57c9f296deb4d40"}
Jul 29 15:13:12.744859 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:12.744806    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gnjvk" podStartSLOduration=129.569143133 podStartE2EDuration="2m10.744785765s" totalImagesPullingTime="1.175642632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:12.743493896 +0000 UTC m=+131.037430704" watchObservedRunningTime="2026-07-29 15:13:12.744785765 +0000 UTC m=+131.038722569"
Jul 29 15:13:21.315518 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.315481    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79d5cbf9db-jjln7"]
Jul 29 15:13:21.335332 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.335304    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.341129 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.341102    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 15:13:21.341545 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.341520    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 15:13:21.341673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.341522    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 15:13:21.342111 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.341941    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 15:13:21.342709 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.342692    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 15:13:21.359094 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.359068    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ftf5b\""
Jul 29 15:13:21.371953 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.371929    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79d5cbf9db-jjln7"]
Jul 29 15:13:21.388657 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.388629    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 15:13:21.443338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.443313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-serving-cert\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.443508 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.443347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-config\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.443508 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.443372    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-trusted-ca-bundle\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.443508 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.443455    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twlrt\" (UniqueName: \"kubernetes.io/projected/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-kube-api-access-twlrt\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.443508 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.443502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-oauth-config\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.443677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.443554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-service-ca\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.443677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.443593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-oauth-serving-cert\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544116 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544088    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-serving-cert\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544236 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-config\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544236 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-trusted-ca-bundle\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544236 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twlrt\" (UniqueName: \"kubernetes.io/projected/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-kube-api-access-twlrt\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544236 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-oauth-config\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544415 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544239    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-service-ca\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544415 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544287    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-oauth-serving-cert\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.544913 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544891    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-config\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.545020 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544934    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-oauth-serving-cert\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.545020 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.544988    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-trusted-ca-bundle\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.545105 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.545030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-service-ca\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.546606 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.546582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-serving-cert\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.546674 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.546584    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-oauth-config\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.555546 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.555518    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twlrt\" (UniqueName: \"kubernetes.io/projected/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-kube-api-access-twlrt\") pod \"console-79d5cbf9db-jjln7\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") " pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.644295 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.644250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:21.765801 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.765745    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-dwq45" event={"ID":"7929f79e-0591-475b-aa42-17f4bc2b24d2","Type":"ContainerStarted","Data":"7ebda000566f269824595de9743acbc59dcf9a525c9014c903f86efdcfc1c0a6"}
Jul 29 15:13:21.766327 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.766136    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-dwq45"
Jul 29 15:13:21.767138 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.767099    2560 patch_prober.go:28] interesting pod/downloads-7c97dc4555-dwq45 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused" start-of-body=
Jul 29 15:13:21.767268 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.767153    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-dwq45" podUID="7929f79e-0591-475b-aa42-17f4bc2b24d2" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused"
Jul 29 15:13:21.788220 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.788141    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-dwq45" podStartSLOduration=1.135817466 podStartE2EDuration="20.788123847s" totalImagesPullingTime="19.652306381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:21.787525435 +0000 UTC m=+140.081462274" watchObservedRunningTime="2026-07-29 15:13:21.788123847 +0000 UTC m=+140.082060649"
Jul 29 15:13:21.818057 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:21.818032    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79d5cbf9db-jjln7"]
Jul 29 15:13:21.820039 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:13:21.820009    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf37c8ca_1131_4890_96b4_c6ff6cb1497a.slice/crio-7a652b73accc1f4c1bc86a77135ed20ed6b1a26bf54c5050a6511312b7ad2b20 WatchSource:0}: Error finding container 7a652b73accc1f4c1bc86a77135ed20ed6b1a26bf54c5050a6511312b7ad2b20: Status 404 returned error can't find the container with id 7a652b73accc1f4c1bc86a77135ed20ed6b1a26bf54c5050a6511312b7ad2b20
Jul 29 15:13:22.773095 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:22.772968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d5cbf9db-jjln7" event={"ID":"bf37c8ca-1131-4890-96b4-c6ff6cb1497a","Type":"ContainerStarted","Data":"7a652b73accc1f4c1bc86a77135ed20ed6b1a26bf54c5050a6511312b7ad2b20"}
Jul 29 15:13:22.786608 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:22.786571    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-dwq45"
Jul 29 15:13:26.794525 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:26.794486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d5cbf9db-jjln7" event={"ID":"bf37c8ca-1131-4890-96b4-c6ff6cb1497a","Type":"ContainerStarted","Data":"8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0"}
Jul 29 15:13:26.818771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:26.818707    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79d5cbf9db-jjln7" podStartSLOduration=1.8452337779999999 podStartE2EDuration="5.81868834s" totalImagesPullingTime="3.973454562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:26.816502646 +0000 UTC m=+145.110439467" watchObservedRunningTime="2026-07-29 15:13:26.81868834 +0000 UTC m=+145.112625141"
Jul 29 15:13:29.049329 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:29.049278    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:29.049914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:29.049423    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:31.645115 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:31.645081    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:31.645626 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:31.645214    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:31.649926 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:31.649904    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:31.818531 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:31.818503    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:13:32.011514 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.011416    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt" podUID="bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" containerName="registry" containerID="cri-o://b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733" gracePeriod=30
Jul 29 15:13:32.281485 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.281416    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:13:32.442119 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442080    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-ca-trust-extracted\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442119 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442122    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-trusted-ca\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442386 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442153    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-bound-sa-token\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442386 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442175    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xwxw\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-kube-api-access-6xwxw\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442386 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442335    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-image-registry-private-configuration\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442619 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442426    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-installation-pull-secrets\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442619 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442483    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-certificates\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442619 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442517    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") pod \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\" (UID: \"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab\") "
Jul 29 15:13:32.442619 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442581    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:13:32.442967 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.442943    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-trusted-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.443085 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.443055    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:13:32.444806 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.444752    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:13:32.444927 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.444871    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:13:32.445008 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.444986    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:13:32.445008 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.444992    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-kube-api-access-6xwxw" (OuterVolumeSpecName: "kube-api-access-6xwxw") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "kube-api-access-6xwxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:13:32.445162 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.445142    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:13:32.451614 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.451584    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" (UID: "bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:13:32.544275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.544203    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-certificates\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.544275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.544233    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-registry-tls\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.544275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.544242    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-ca-trust-extracted\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.544275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.544251    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-bound-sa-token\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.544275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.544260    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xwxw\" (UniqueName: \"kubernetes.io/projected/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-kube-api-access-6xwxw\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.544275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.544269    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-image-registry-private-configuration\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.544275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.544278    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab-installation-pull-secrets\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:13:32.819343 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.819308    2560 generic.go:356] "Generic (PLEG): container finished" podID="bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" containerID="b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733" exitCode=0
Jul 29 15:13:32.819696 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.819369    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt"
Jul 29 15:13:32.819696 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.819389    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt" event={"ID":"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab","Type":"ContainerDied","Data":"b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733"}
Jul 29 15:13:32.819696 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.819480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d6d695b6d-9gwgt" event={"ID":"bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab","Type":"ContainerDied","Data":"e864e4821946a8f84e9a1e672b30869046e46715e3fb0fa9acfe7208cb93806a"}
Jul 29 15:13:32.819696 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.819508    2560 scope.go:117] "RemoveContainer" containerID="b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733"
Jul 29 15:13:32.828320 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.828301    2560 scope.go:117] "RemoveContainer" containerID="b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733"
Jul 29 15:13:32.828641 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:13:32.828619    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733\": container with ID starting with b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733 not found: ID does not exist" containerID="b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733"
Jul 29 15:13:32.828697 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.828652    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733"} err="failed to get container status \"b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733\": rpc error: code = NotFound desc = could not find container \"b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733\": container with ID starting with b314c9df991218afc4684b689a9f699aa9dba37ff91be203ff0cb4678fad4733 not found: ID does not exist"
Jul 29 15:13:32.844219 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.844194    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d6d695b6d-9gwgt"]
Jul 29 15:13:32.851548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:32.851525    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d6d695b6d-9gwgt"]
Jul 29 15:13:33.825929 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:33.825899    2560 generic.go:356] "Generic (PLEG): container finished" podID="96198947-8a41-49ad-baeb-c8d609ef2cb8" containerID="02494d802bbab9ac306fc4ee3f80851032461c8ec6b206c344ce971415c97031" exitCode=0
Jul 29 15:13:33.826321 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:33.825968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5" event={"ID":"96198947-8a41-49ad-baeb-c8d609ef2cb8","Type":"ContainerDied","Data":"02494d802bbab9ac306fc4ee3f80851032461c8ec6b206c344ce971415c97031"}
Jul 29 15:13:33.826400 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:33.826385    2560 scope.go:117] "RemoveContainer" containerID="02494d802bbab9ac306fc4ee3f80851032461c8ec6b206c344ce971415c97031"
Jul 29 15:13:34.165587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:34.165552    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" path="/var/lib/kubelet/pods/bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab/volumes"
Jul 29 15:13:34.831799 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:34.831768    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5" event={"ID":"96198947-8a41-49ad-baeb-c8d609ef2cb8","Type":"ContainerStarted","Data":"8aac122f401d4386adf2f4a030469b45a1045299f0b1af5247ab11ef2ff87184"}
Jul 29 15:13:49.054683 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:49.054650    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:13:49.058472 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:13:49.058452    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-67bf84cdff-rkndn"
Jul 29 15:14:47.361873 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.361841    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5959f77cdf-9wxcv"]
Jul 29 15:14:47.362378 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.362134    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" containerName="registry"
Jul 29 15:14:47.362378 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.362145    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" containerName="registry"
Jul 29 15:14:47.362378 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.362213    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf77d19c-fa4d-41aa-9d0b-3ed1a7c49bab" containerName="registry"
Jul 29 15:14:47.365112 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.365096    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.387600 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.387570    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5959f77cdf-9wxcv"]
Jul 29 15:14:47.438059 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.438024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-oauth-serving-cert\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.438059 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.438061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-oauth-config\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.438324 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.438093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-service-ca\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.438324 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.438171    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-serving-cert\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.438324 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.438228    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-console-config\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.438324 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.438246    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6748k\" (UniqueName: \"kubernetes.io/projected/484d3aa4-0026-4593-a538-b234269800ec-kube-api-access-6748k\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.438324 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.438291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-trusted-ca-bundle\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.538865 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.538824    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-trusted-ca-bundle\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.538865 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.538874    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-oauth-serving-cert\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.538895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-oauth-config\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.538940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-service-ca\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.538977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-serving-cert\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.539009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-console-config\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.539032    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6748k\" (UniqueName: \"kubernetes.io/projected/484d3aa4-0026-4593-a538-b234269800ec-kube-api-access-6748k\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539745 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.539717    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-oauth-serving-cert\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539861 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.539790    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-service-ca\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539861 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.539808    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-trusted-ca-bundle\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.539861 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.539793    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-console-config\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.541537 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.541512    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-oauth-config\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.541627 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.541545    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-serving-cert\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.550520 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.550502    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6748k\" (UniqueName: \"kubernetes.io/projected/484d3aa4-0026-4593-a538-b234269800ec-kube-api-access-6748k\") pod \"console-5959f77cdf-9wxcv\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") " pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.674136 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.674044    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:47.806013 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:47.805976    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5959f77cdf-9wxcv"]
Jul 29 15:14:47.808960 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:14:47.808931    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484d3aa4_0026_4593_a538_b234269800ec.slice/crio-871de3f797ad75c661a9daf02fa0b74ab362220d77a993981c7e97a674ff30a5 WatchSource:0}: Error finding container 871de3f797ad75c661a9daf02fa0b74ab362220d77a993981c7e97a674ff30a5: Status 404 returned error can't find the container with id 871de3f797ad75c661a9daf02fa0b74ab362220d77a993981c7e97a674ff30a5
Jul 29 15:14:48.104025 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:48.103986    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5959f77cdf-9wxcv" event={"ID":"484d3aa4-0026-4593-a538-b234269800ec","Type":"ContainerStarted","Data":"df9ad791d3357699888e95a6ffdd4b05e38868900682cf743a42fd0e88f79f34"}
Jul 29 15:14:48.104229 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:48.104042    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5959f77cdf-9wxcv" event={"ID":"484d3aa4-0026-4593-a538-b234269800ec","Type":"ContainerStarted","Data":"871de3f797ad75c661a9daf02fa0b74ab362220d77a993981c7e97a674ff30a5"}
Jul 29 15:14:48.132487 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:48.132431    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5959f77cdf-9wxcv" podStartSLOduration=1.132389998 podStartE2EDuration="1.132389998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:14:48.131486646 +0000 UTC m=+226.425423472" watchObservedRunningTime="2026-07-29 15:14:48.132389998 +0000 UTC m=+226.426326800"
Jul 29 15:14:57.674919 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:57.674873    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:57.675351 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:57.674950    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:57.679771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:57.679749    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:58.145565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:58.145539    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:14:58.198882 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:14:58.198847    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79d5cbf9db-jjln7"]
Jul 29 15:15:23.218138 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.218085    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79d5cbf9db-jjln7" podUID="bf37c8ca-1131-4890-96b4-c6ff6cb1497a" containerName="console" containerID="cri-o://8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0" gracePeriod=15
Jul 29 15:15:23.463138 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.463112    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:15:23.525981 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.525890    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-serving-cert\") pod \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") "
Jul 29 15:15:23.525981 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.525932    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-trusted-ca-bundle\") pod \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") "
Jul 29 15:15:23.526203 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.525995    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twlrt\" (UniqueName: \"kubernetes.io/projected/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-kube-api-access-twlrt\") pod \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") "
Jul 29 15:15:23.526203 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526031    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-config\") pod \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") "
Jul 29 15:15:23.526203 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526077    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-service-ca\") pod \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") "
Jul 29 15:15:23.526203 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526106    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-oauth-config\") pod \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") "
Jul 29 15:15:23.526203 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526134    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-oauth-serving-cert\") pod \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\" (UID: \"bf37c8ca-1131-4890-96b4-c6ff6cb1497a\") "
Jul 29 15:15:23.526476 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526433    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bf37c8ca-1131-4890-96b4-c6ff6cb1497a" (UID: "bf37c8ca-1131-4890-96b4-c6ff6cb1497a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:15:23.526539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526499    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-service-ca" (OuterVolumeSpecName: "service-ca") pod "bf37c8ca-1131-4890-96b4-c6ff6cb1497a" (UID: "bf37c8ca-1131-4890-96b4-c6ff6cb1497a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:15:23.526639 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526504    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-config" (OuterVolumeSpecName: "console-config") pod "bf37c8ca-1131-4890-96b4-c6ff6cb1497a" (UID: "bf37c8ca-1131-4890-96b4-c6ff6cb1497a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:15:23.526771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.526686    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bf37c8ca-1131-4890-96b4-c6ff6cb1497a" (UID: "bf37c8ca-1131-4890-96b4-c6ff6cb1497a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:15:23.528411 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.528383    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bf37c8ca-1131-4890-96b4-c6ff6cb1497a" (UID: "bf37c8ca-1131-4890-96b4-c6ff6cb1497a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:15:23.528537 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.528427    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bf37c8ca-1131-4890-96b4-c6ff6cb1497a" (UID: "bf37c8ca-1131-4890-96b4-c6ff6cb1497a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:15:23.528537 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.528428    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-kube-api-access-twlrt" (OuterVolumeSpecName: "kube-api-access-twlrt") pod "bf37c8ca-1131-4890-96b4-c6ff6cb1497a" (UID: "bf37c8ca-1131-4890-96b4-c6ff6cb1497a"). InnerVolumeSpecName "kube-api-access-twlrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:15:23.626910 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.626871    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-serving-cert\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:23.626910 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.626903    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-trusted-ca-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:23.626910 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.626913    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twlrt\" (UniqueName: \"kubernetes.io/projected/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-kube-api-access-twlrt\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:23.627144 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.626923    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-config\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:23.627144 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.626932    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:23.627144 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.626939    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-console-oauth-config\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:23.627144 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:23.626948    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf37c8ca-1131-4890-96b4-c6ff6cb1497a-oauth-serving-cert\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:24.244118 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.244084    2560 generic.go:356] "Generic (PLEG): container finished" podID="bf37c8ca-1131-4890-96b4-c6ff6cb1497a" containerID="8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0" exitCode=0
Jul 29 15:15:24.244613 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.244134    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d5cbf9db-jjln7" event={"ID":"bf37c8ca-1131-4890-96b4-c6ff6cb1497a","Type":"ContainerDied","Data":"8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0"}
Jul 29 15:15:24.244613 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.244156    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d5cbf9db-jjln7"
Jul 29 15:15:24.244613 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.244226    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d5cbf9db-jjln7" event={"ID":"bf37c8ca-1131-4890-96b4-c6ff6cb1497a","Type":"ContainerDied","Data":"7a652b73accc1f4c1bc86a77135ed20ed6b1a26bf54c5050a6511312b7ad2b20"}
Jul 29 15:15:24.244613 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.244253    2560 scope.go:117] "RemoveContainer" containerID="8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0"
Jul 29 15:15:24.252285 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.252267    2560 scope.go:117] "RemoveContainer" containerID="8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0"
Jul 29 15:15:24.252570 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:15:24.252536    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0\": container with ID starting with 8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0 not found: ID does not exist" containerID="8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0"
Jul 29 15:15:24.252622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.252581    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0"} err="failed to get container status \"8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0\": rpc error: code = NotFound desc = could not find container \"8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0\": container with ID starting with 8a2a0ab70dc848ca97e97c4a3a76e375c87c1e52a21bbab2105a3cee205b0dd0 not found: ID does not exist"
Jul 29 15:15:24.265504 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.265476    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79d5cbf9db-jjln7"]
Jul 29 15:15:24.271396 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:24.271370    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79d5cbf9db-jjln7"]
Jul 29 15:15:26.165196 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:26.165162    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf37c8ca-1131-4890-96b4-c6ff6cb1497a" path="/var/lib/kubelet/pods/bf37c8ca-1131-4890-96b4-c6ff6cb1497a/volumes"
Jul 29 15:15:31.415281 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.415246    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"]
Jul 29 15:15:31.415671 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.415565    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf37c8ca-1131-4890-96b4-c6ff6cb1497a" containerName="console"
Jul 29 15:15:31.415671 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.415576    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf37c8ca-1131-4890-96b4-c6ff6cb1497a" containerName="console"
Jul 29 15:15:31.415671 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.415638    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf37c8ca-1131-4890-96b4-c6ff6cb1497a" containerName="console"
Jul 29 15:15:31.420123 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.420103    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.423609 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.423588    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 15:15:31.424094 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.424073    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 15:15:31.424541 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.424526    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kmgkh\""
Jul 29 15:15:31.434496 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.434465    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"]
Jul 29 15:15:31.490154 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.490118    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.490373 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.490166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44sqs\" (UniqueName: \"kubernetes.io/projected/797f68d1-85d1-4b13-a192-d93ae7bbee5a-kube-api-access-44sqs\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.490373 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.490297    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.591617 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.591565    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.591771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.591628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.591771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.591656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44sqs\" (UniqueName: \"kubernetes.io/projected/797f68d1-85d1-4b13-a192-d93ae7bbee5a-kube-api-access-44sqs\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.591964 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.591944    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.592000 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.591976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.601692 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.601661    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44sqs\" (UniqueName: \"kubernetes.io/projected/797f68d1-85d1-4b13-a192-d93ae7bbee5a-kube-api-access-44sqs\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.729843 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.729734    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:31.869107 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:31.868998    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"]
Jul 29 15:15:31.871764 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:15:31.871736    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod797f68d1_85d1_4b13_a192_d93ae7bbee5a.slice/crio-9f49d801c28d592abc9cdff676f7f4374d83f77a2722d54f8b45b48d10b61524 WatchSource:0}: Error finding container 9f49d801c28d592abc9cdff676f7f4374d83f77a2722d54f8b45b48d10b61524: Status 404 returned error can't find the container with id 9f49d801c28d592abc9cdff676f7f4374d83f77a2722d54f8b45b48d10b61524
Jul 29 15:15:32.275146 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:32.275108    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds" event={"ID":"797f68d1-85d1-4b13-a192-d93ae7bbee5a","Type":"ContainerStarted","Data":"9f49d801c28d592abc9cdff676f7f4374d83f77a2722d54f8b45b48d10b61524"}
Jul 29 15:15:38.304613 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:38.304578    2560 generic.go:356] "Generic (PLEG): container finished" podID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerID="b0efeb61687214280eb49c93fd038e6e4915305d0e9f4779c35b54962324d369" exitCode=0
Jul 29 15:15:38.305095 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:38.304635    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds" event={"ID":"797f68d1-85d1-4b13-a192-d93ae7bbee5a","Type":"ContainerDied","Data":"b0efeb61687214280eb49c93fd038e6e4915305d0e9f4779c35b54962324d369"}
Jul 29 15:15:41.317556 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:41.317508    2560 generic.go:356] "Generic (PLEG): container finished" podID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerID="cb747cdaef25c6965c97e9250d9d67bf8b366b3cca5d0fa01b13b7fc39d88071" exitCode=0
Jul 29 15:15:41.317953 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:41.317584    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds" event={"ID":"797f68d1-85d1-4b13-a192-d93ae7bbee5a","Type":"ContainerDied","Data":"cb747cdaef25c6965c97e9250d9d67bf8b366b3cca5d0fa01b13b7fc39d88071"}
Jul 29 15:15:49.355264 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:49.355228    2560 generic.go:356] "Generic (PLEG): container finished" podID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerID="696fadc2154d167c70bdb61e17959a9ff3dd2090f8d38256d9024b27e669a91a" exitCode=0
Jul 29 15:15:49.355681 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:49.355311    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds" event={"ID":"797f68d1-85d1-4b13-a192-d93ae7bbee5a","Type":"ContainerDied","Data":"696fadc2154d167c70bdb61e17959a9ff3dd2090f8d38256d9024b27e669a91a"}
Jul 29 15:15:50.487090 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.487065    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:50.666436 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.666341    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-bundle\") pod \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") "
Jul 29 15:15:50.666436 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.666387    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44sqs\" (UniqueName: \"kubernetes.io/projected/797f68d1-85d1-4b13-a192-d93ae7bbee5a-kube-api-access-44sqs\") pod \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") "
Jul 29 15:15:50.666622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.666455    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-util\") pod \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\" (UID: \"797f68d1-85d1-4b13-a192-d93ae7bbee5a\") "
Jul 29 15:15:50.667237 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.667174    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-bundle" (OuterVolumeSpecName: "bundle") pod "797f68d1-85d1-4b13-a192-d93ae7bbee5a" (UID: "797f68d1-85d1-4b13-a192-d93ae7bbee5a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:15:50.668678 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.668650    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/797f68d1-85d1-4b13-a192-d93ae7bbee5a-kube-api-access-44sqs" (OuterVolumeSpecName: "kube-api-access-44sqs") pod "797f68d1-85d1-4b13-a192-d93ae7bbee5a" (UID: "797f68d1-85d1-4b13-a192-d93ae7bbee5a"). InnerVolumeSpecName "kube-api-access-44sqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:15:50.671324 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.671296    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-util" (OuterVolumeSpecName: "util") pod "797f68d1-85d1-4b13-a192-d93ae7bbee5a" (UID: "797f68d1-85d1-4b13-a192-d93ae7bbee5a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:15:50.767605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.767565    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:50.767605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.767601    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44sqs\" (UniqueName: \"kubernetes.io/projected/797f68d1-85d1-4b13-a192-d93ae7bbee5a-kube-api-access-44sqs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:50.767605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:50.767611    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797f68d1-85d1-4b13-a192-d93ae7bbee5a-util\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:15:51.366292 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:51.366258    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds" event={"ID":"797f68d1-85d1-4b13-a192-d93ae7bbee5a","Type":"ContainerDied","Data":"9f49d801c28d592abc9cdff676f7f4374d83f77a2722d54f8b45b48d10b61524"}
Jul 29 15:15:51.366466 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:51.366305    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f49d801c28d592abc9cdff676f7f4374d83f77a2722d54f8b45b48d10b61524"
Jul 29 15:15:51.366466 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:51.366308    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bpsds"
Jul 29 15:15:54.561161 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561125    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"]
Jul 29 15:15:54.561677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561587    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerName="util"
Jul 29 15:15:54.561677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561606    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerName="util"
Jul 29 15:15:54.561677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561626    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerName="extract"
Jul 29 15:15:54.561677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561634    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerName="extract"
Jul 29 15:15:54.561677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561646    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerName="pull"
Jul 29 15:15:54.561677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561654    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerName="pull"
Jul 29 15:15:54.562013 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.561737    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="797f68d1-85d1-4b13-a192-d93ae7bbee5a" containerName="extract"
Jul 29 15:15:54.564748 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.564726    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:54.568243 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.568222    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 15:15:54.568455 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.568436    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:15:54.568538 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.568520    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f7d2m\""
Jul 29 15:15:54.587895 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.587868    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"]
Jul 29 15:15:54.595777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.595751    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7268e753-7cc4-41d9-afe1-e87a28ff56e8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cq8jr\" (UID: \"7268e753-7cc4-41d9-afe1-e87a28ff56e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:54.595906 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.595798    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mpzp\" (UniqueName: \"kubernetes.io/projected/7268e753-7cc4-41d9-afe1-e87a28ff56e8-kube-api-access-5mpzp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cq8jr\" (UID: \"7268e753-7cc4-41d9-afe1-e87a28ff56e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:54.696776 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.696732    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mpzp\" (UniqueName: \"kubernetes.io/projected/7268e753-7cc4-41d9-afe1-e87a28ff56e8-kube-api-access-5mpzp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cq8jr\" (UID: \"7268e753-7cc4-41d9-afe1-e87a28ff56e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:54.696972 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.696829    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7268e753-7cc4-41d9-afe1-e87a28ff56e8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cq8jr\" (UID: \"7268e753-7cc4-41d9-afe1-e87a28ff56e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:54.697238 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.697219    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7268e753-7cc4-41d9-afe1-e87a28ff56e8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cq8jr\" (UID: \"7268e753-7cc4-41d9-afe1-e87a28ff56e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:54.708220 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.708179    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mpzp\" (UniqueName: \"kubernetes.io/projected/7268e753-7cc4-41d9-afe1-e87a28ff56e8-kube-api-access-5mpzp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cq8jr\" (UID: \"7268e753-7cc4-41d9-afe1-e87a28ff56e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:54.874820 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:54.874787    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"
Jul 29 15:15:55.010138 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:55.010108    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr"]
Jul 29 15:15:55.013976 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:15:55.013945    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7268e753_7cc4_41d9_afe1_e87a28ff56e8.slice/crio-b0f7ea6bf70f5fe0787fe1e87fd00fcae772e82ea7cef0277e7ba2ffee6d534a WatchSource:0}: Error finding container b0f7ea6bf70f5fe0787fe1e87fd00fcae772e82ea7cef0277e7ba2ffee6d534a: Status 404 returned error can't find the container with id b0f7ea6bf70f5fe0787fe1e87fd00fcae772e82ea7cef0277e7ba2ffee6d534a
Jul 29 15:15:55.385430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:55.385393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr" event={"ID":"7268e753-7cc4-41d9-afe1-e87a28ff56e8","Type":"ContainerStarted","Data":"b0f7ea6bf70f5fe0787fe1e87fd00fcae772e82ea7cef0277e7ba2ffee6d534a"}
Jul 29 15:15:59.404244 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:59.404201    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr" event={"ID":"7268e753-7cc4-41d9-afe1-e87a28ff56e8","Type":"ContainerStarted","Data":"1863f6cd0956fb6b543ea7109ca67b9950d16169d3193a9efc82cf6d2a211b1d"}
Jul 29 15:15:59.429836 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:15:59.429773    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cq8jr" podStartSLOduration=1.678584962 podStartE2EDuration="5.429749632s" totalImagesPullingTime="3.75116467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:15:59.428863767 +0000 UTC m=+297.722800593" watchObservedRunningTime="2026-07-29 15:15:59.429749632 +0000 UTC m=+297.723686435"
Jul 29 15:16:00.134330 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.134293    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xxsbl"]
Jul 29 15:16:00.137666 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.137648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.140719 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.140694    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-47shz\""
Jul 29 15:16:00.140719 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.140714    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 15:16:00.142451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.142431    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 15:16:00.150097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.150070    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xxsbl"]
Jul 29 15:16:00.240853 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.240815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xxsbl\" (UID: \"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.241070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.240898    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd6z9\" (UniqueName: \"kubernetes.io/projected/4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1-kube-api-access-xd6z9\") pod \"cert-manager-webhook-6486ffd997-xxsbl\" (UID: \"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.342011 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.341969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd6z9\" (UniqueName: \"kubernetes.io/projected/4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1-kube-api-access-xd6z9\") pod \"cert-manager-webhook-6486ffd997-xxsbl\" (UID: \"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.342255 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.342033    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xxsbl\" (UID: \"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.355449 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.355417    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xxsbl\" (UID: \"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.355605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.355523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd6z9\" (UniqueName: \"kubernetes.io/projected/4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1-kube-api-access-xd6z9\") pod \"cert-manager-webhook-6486ffd997-xxsbl\" (UID: \"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.447222 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.447110    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:00.571750 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.571718    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"]
Jul 29 15:16:00.576406 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.576382    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.579227 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.579197    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 15:16:00.579365 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.579211    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 15:16:00.579365 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.579279    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kmgkh\""
Jul 29 15:16:00.585984 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.585955    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"]
Jul 29 15:16:00.591884 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.591721    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xxsbl"]
Jul 29 15:16:00.594672 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:16:00.594645    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b3c4e61_0c20_4c5c_b3d5_3db7a5a25bd1.slice/crio-9ae532e05ae29c165c7e8555d94bdd6d0c695a9bcf2a7c8ad45a330f40f8728d WatchSource:0}: Error finding container 9ae532e05ae29c165c7e8555d94bdd6d0c695a9bcf2a7c8ad45a330f40f8728d: Status 404 returned error can't find the container with id 9ae532e05ae29c165c7e8555d94bdd6d0c695a9bcf2a7c8ad45a330f40f8728d
Jul 29 15:16:00.643977 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.643934    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.643977 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.643981    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.644216 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.644022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zjv6\" (UniqueName: \"kubernetes.io/projected/b81a6b23-ff5d-4735-ace9-1456545722b0-kube-api-access-5zjv6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.744873 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.744767    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zjv6\" (UniqueName: \"kubernetes.io/projected/b81a6b23-ff5d-4735-ace9-1456545722b0-kube-api-access-5zjv6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.745061 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.744870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.745061 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.744921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.745422 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.745397    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.745505 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.745397    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.753996 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.753953    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zjv6\" (UniqueName: \"kubernetes.io/projected/b81a6b23-ff5d-4735-ace9-1456545722b0-kube-api-access-5zjv6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:00.887209 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:00.887141    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:01.023785 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:01.023757    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"]
Jul 29 15:16:01.025738 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:16:01.025704    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81a6b23_ff5d_4735_ace9_1456545722b0.slice/crio-4e264a92601d0fa8781e62e49cca53f46444576f1c8a5ec1f438491d2331a677 WatchSource:0}: Error finding container 4e264a92601d0fa8781e62e49cca53f46444576f1c8a5ec1f438491d2331a677: Status 404 returned error can't find the container with id 4e264a92601d0fa8781e62e49cca53f46444576f1c8a5ec1f438491d2331a677
Jul 29 15:16:01.416840 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:01.416798    2560 generic.go:356] "Generic (PLEG): container finished" podID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerID="cd0b78bb8e0a99c0881267cc0deb09225d505b02e9098377387bccbf4728ef74" exitCode=0
Jul 29 15:16:01.417055 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:01.416896    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz" event={"ID":"b81a6b23-ff5d-4735-ace9-1456545722b0","Type":"ContainerDied","Data":"cd0b78bb8e0a99c0881267cc0deb09225d505b02e9098377387bccbf4728ef74"}
Jul 29 15:16:01.417055 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:01.416958    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz" event={"ID":"b81a6b23-ff5d-4735-ace9-1456545722b0","Type":"ContainerStarted","Data":"4e264a92601d0fa8781e62e49cca53f46444576f1c8a5ec1f438491d2331a677"}
Jul 29 15:16:01.418587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:01.418510    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl" event={"ID":"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1","Type":"ContainerStarted","Data":"9ae532e05ae29c165c7e8555d94bdd6d0c695a9bcf2a7c8ad45a330f40f8728d"}
Jul 29 15:16:02.036333 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:02.036302    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:16:02.036818 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:02.036682    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:16:02.050426 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:02.050403    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:16:02.050581 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:02.050439    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:16:02.055222 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:02.055202    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 15:16:04.435579 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:04.435470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl" event={"ID":"4b3c4e61-0c20-4c5c-b3d5-3db7a5a25bd1","Type":"ContainerStarted","Data":"87537695d348a62d0d68c695190ac9e3cf1d093739d4b7656997f0b1e9dddc3c"}
Jul 29 15:16:04.435579 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:04.435525    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:04.437350 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:04.437328    2560 generic.go:356] "Generic (PLEG): container finished" podID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerID="24796d9fe304ca262a1e4acaf98506e317ce37dd8147c8f98b2f3db2d70c65b3" exitCode=0
Jul 29 15:16:04.437430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:04.437402    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz" event={"ID":"b81a6b23-ff5d-4735-ace9-1456545722b0","Type":"ContainerDied","Data":"24796d9fe304ca262a1e4acaf98506e317ce37dd8147c8f98b2f3db2d70c65b3"}
Jul 29 15:16:04.452409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:04.452362    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl" podStartSLOduration=0.906561442 podStartE2EDuration="4.452342023s" totalImagesPullingTime="3.545780581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:16:04.451622454 +0000 UTC m=+302.745559278" watchObservedRunningTime="2026-07-29 15:16:04.452342023 +0000 UTC m=+302.746278826"
Jul 29 15:16:05.443478 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:05.443444    2560 generic.go:356] "Generic (PLEG): container finished" podID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerID="8146dec1f21398904ae326681b5ba7fddc612f7ea9134b62ca65dcdf97a82e24" exitCode=0
Jul 29 15:16:05.443886 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:05.443495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz" event={"ID":"b81a6b23-ff5d-4735-ace9-1456545722b0","Type":"ContainerDied","Data":"8146dec1f21398904ae326681b5ba7fddc612f7ea9134b62ca65dcdf97a82e24"}
Jul 29 15:16:06.570306 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.570281    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:06.599262 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.599233    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-util\") pod \"b81a6b23-ff5d-4735-ace9-1456545722b0\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") "
Jul 29 15:16:06.599442 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.599274    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zjv6\" (UniqueName: \"kubernetes.io/projected/b81a6b23-ff5d-4735-ace9-1456545722b0-kube-api-access-5zjv6\") pod \"b81a6b23-ff5d-4735-ace9-1456545722b0\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") "
Jul 29 15:16:06.599442 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.599340    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-bundle\") pod \"b81a6b23-ff5d-4735-ace9-1456545722b0\" (UID: \"b81a6b23-ff5d-4735-ace9-1456545722b0\") "
Jul 29 15:16:06.599842 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.599815    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-bundle" (OuterVolumeSpecName: "bundle") pod "b81a6b23-ff5d-4735-ace9-1456545722b0" (UID: "b81a6b23-ff5d-4735-ace9-1456545722b0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:16:06.602084 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.602060    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b81a6b23-ff5d-4735-ace9-1456545722b0-kube-api-access-5zjv6" (OuterVolumeSpecName: "kube-api-access-5zjv6") pod "b81a6b23-ff5d-4735-ace9-1456545722b0" (UID: "b81a6b23-ff5d-4735-ace9-1456545722b0"). InnerVolumeSpecName "kube-api-access-5zjv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:16:06.700338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.700247    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zjv6\" (UniqueName: \"kubernetes.io/projected/b81a6b23-ff5d-4735-ace9-1456545722b0-kube-api-access-5zjv6\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:16:06.700338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.700279    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:16:06.703212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.703174    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-util" (OuterVolumeSpecName: "util") pod "b81a6b23-ff5d-4735-ace9-1456545722b0" (UID: "b81a6b23-ff5d-4735-ace9-1456545722b0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:16:06.801382 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:06.801341    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b81a6b23-ff5d-4735-ace9-1456545722b0-util\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:16:07.093081 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093054    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"]
Jul 29 15:16:07.093394 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093381    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerName="pull"
Jul 29 15:16:07.093394 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093394    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerName="pull"
Jul 29 15:16:07.093480 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093406    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerName="util"
Jul 29 15:16:07.093480 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093412    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerName="util"
Jul 29 15:16:07.093480 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093419    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerName="extract"
Jul 29 15:16:07.093480 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093425    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerName="extract"
Jul 29 15:16:07.093596 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.093481    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b81a6b23-ff5d-4735-ace9-1456545722b0" containerName="extract"
Jul 29 15:16:07.097729 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.097710    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.099871 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.099848    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5v95l\""
Jul 29 15:16:07.107166 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.107144    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"]
Jul 29 15:16:07.204526 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.204491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f62966b2-4b90-49f2-94c1-a31c952567c2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-l8sdh\" (UID: \"f62966b2-4b90-49f2-94c1-a31c952567c2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.204711 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.204543    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc5sr\" (UniqueName: \"kubernetes.io/projected/f62966b2-4b90-49f2-94c1-a31c952567c2-kube-api-access-qc5sr\") pod \"cert-manager-cainjector-559cdd8c87-l8sdh\" (UID: \"f62966b2-4b90-49f2-94c1-a31c952567c2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.305744 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.305689    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f62966b2-4b90-49f2-94c1-a31c952567c2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-l8sdh\" (UID: \"f62966b2-4b90-49f2-94c1-a31c952567c2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.305940 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.305778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc5sr\" (UniqueName: \"kubernetes.io/projected/f62966b2-4b90-49f2-94c1-a31c952567c2-kube-api-access-qc5sr\") pod \"cert-manager-cainjector-559cdd8c87-l8sdh\" (UID: \"f62966b2-4b90-49f2-94c1-a31c952567c2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.315384 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.315354    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f62966b2-4b90-49f2-94c1-a31c952567c2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-l8sdh\" (UID: \"f62966b2-4b90-49f2-94c1-a31c952567c2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.315510 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.315459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc5sr\" (UniqueName: \"kubernetes.io/projected/f62966b2-4b90-49f2-94c1-a31c952567c2-kube-api-access-qc5sr\") pod \"cert-manager-cainjector-559cdd8c87-l8sdh\" (UID: \"f62966b2-4b90-49f2-94c1-a31c952567c2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.406800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.406717    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"
Jul 29 15:16:07.455539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.455480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz" event={"ID":"b81a6b23-ff5d-4735-ace9-1456545722b0","Type":"ContainerDied","Data":"4e264a92601d0fa8781e62e49cca53f46444576f1c8a5ec1f438491d2331a677"}
Jul 29 15:16:07.455539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.455523    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmr4tz"
Jul 29 15:16:07.455539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.455540    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e264a92601d0fa8781e62e49cca53f46444576f1c8a5ec1f438491d2331a677"
Jul 29 15:16:07.540521 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:07.540496    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh"]
Jul 29 15:16:07.542803 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:16:07.542772    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf62966b2_4b90_49f2_94c1_a31c952567c2.slice/crio-65c562fb14b306357bd1dbd40679d5ec011d92ffd6b9cb4445058613da3d6781 WatchSource:0}: Error finding container 65c562fb14b306357bd1dbd40679d5ec011d92ffd6b9cb4445058613da3d6781: Status 404 returned error can't find the container with id 65c562fb14b306357bd1dbd40679d5ec011d92ffd6b9cb4445058613da3d6781
Jul 29 15:16:08.461595 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:08.461556    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh" event={"ID":"f62966b2-4b90-49f2-94c1-a31c952567c2","Type":"ContainerStarted","Data":"f2447e8673640b855034d8f20c67f6e4590c67957710a98c45ca099e25e69c7d"}
Jul 29 15:16:08.461595 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:08.461599    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh" event={"ID":"f62966b2-4b90-49f2-94c1-a31c952567c2","Type":"ContainerStarted","Data":"65c562fb14b306357bd1dbd40679d5ec011d92ffd6b9cb4445058613da3d6781"}
Jul 29 15:16:08.480311 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:08.480252    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l8sdh" podStartSLOduration=1.480235548 podStartE2EDuration="1.480235548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:16:08.478515549 +0000 UTC m=+306.772452351" watchObservedRunningTime="2026-07-29 15:16:08.480235548 +0000 UTC m=+306.774172350"
Jul 29 15:16:10.446452 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:10.446412    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xxsbl"
Jul 29 15:16:17.317894 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.317856    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"]
Jul 29 15:16:17.360224 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.360163    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"]
Jul 29 15:16:17.360404 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.360277    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.363016 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.362992    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 15:16:17.363161 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.362999    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kmgkh\""
Jul 29 15:16:17.363984 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.363967    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 15:16:17.497416 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.497381    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.497584 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.497425    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.497584 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.497538    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nsdp\" (UniqueName: \"kubernetes.io/projected/20528816-15a2-49af-9dc9-88972c4ca653-kube-api-access-5nsdp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.598194 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.598161    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.598342 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.598290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nsdp\" (UniqueName: \"kubernetes.io/projected/20528816-15a2-49af-9dc9-88972c4ca653-kube-api-access-5nsdp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.598384 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.598344    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.598593 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.598567    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.598635 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.598613    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.607947 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.607923    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nsdp\" (UniqueName: \"kubernetes.io/projected/20528816-15a2-49af-9dc9-88972c4ca653-kube-api-access-5nsdp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.669877 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.669842    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:17.820950 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:17.820925    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"]
Jul 29 15:16:17.823410 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:16:17.823374    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20528816_15a2_49af_9dc9_88972c4ca653.slice/crio-3af745dc5e6cf9e333f5eb70b26033e424013783030449b451aeb7b5daccbb00 WatchSource:0}: Error finding container 3af745dc5e6cf9e333f5eb70b26033e424013783030449b451aeb7b5daccbb00: Status 404 returned error can't find the container with id 3af745dc5e6cf9e333f5eb70b26033e424013783030449b451aeb7b5daccbb00
Jul 29 15:16:18.506777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:18.506738    2560 generic.go:356] "Generic (PLEG): container finished" podID="20528816-15a2-49af-9dc9-88972c4ca653" containerID="6058d437dea3f690eef4889d779199028390867082dcefebddf07601be4b5b43" exitCode=0
Jul 29 15:16:18.507233 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:18.506823    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp" event={"ID":"20528816-15a2-49af-9dc9-88972c4ca653","Type":"ContainerDied","Data":"6058d437dea3f690eef4889d779199028390867082dcefebddf07601be4b5b43"}
Jul 29 15:16:18.507233 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:18.506867    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp" event={"ID":"20528816-15a2-49af-9dc9-88972c4ca653","Type":"ContainerStarted","Data":"3af745dc5e6cf9e333f5eb70b26033e424013783030449b451aeb7b5daccbb00"}
Jul 29 15:16:18.507326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:18.507305    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:16:20.522008 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:20.521971    2560 generic.go:356] "Generic (PLEG): container finished" podID="20528816-15a2-49af-9dc9-88972c4ca653" containerID="a633138d2c709ad1125b48a622509db99fa02eea03199aa8b795bb8155d9411e" exitCode=0
Jul 29 15:16:20.522518 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:20.522061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp" event={"ID":"20528816-15a2-49af-9dc9-88972c4ca653","Type":"ContainerDied","Data":"a633138d2c709ad1125b48a622509db99fa02eea03199aa8b795bb8155d9411e"}
Jul 29 15:16:21.529125 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:21.529090    2560 generic.go:356] "Generic (PLEG): container finished" podID="20528816-15a2-49af-9dc9-88972c4ca653" containerID="c84f7b476bd0e91dc1d7c9044b4679cb36e1ebbd5d02caf801b91c6782c5ebd8" exitCode=0
Jul 29 15:16:21.529546 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:21.529135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp" event={"ID":"20528816-15a2-49af-9dc9-88972c4ca653","Type":"ContainerDied","Data":"c84f7b476bd0e91dc1d7c9044b4679cb36e1ebbd5d02caf801b91c6782c5ebd8"}
Jul 29 15:16:22.673085 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.673058    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:22.741784 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.741752    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nsdp\" (UniqueName: \"kubernetes.io/projected/20528816-15a2-49af-9dc9-88972c4ca653-kube-api-access-5nsdp\") pod \"20528816-15a2-49af-9dc9-88972c4ca653\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") "
Jul 29 15:16:22.741949 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.741860    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-bundle\") pod \"20528816-15a2-49af-9dc9-88972c4ca653\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") "
Jul 29 15:16:22.741949 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.741931    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-util\") pod \"20528816-15a2-49af-9dc9-88972c4ca653\" (UID: \"20528816-15a2-49af-9dc9-88972c4ca653\") "
Jul 29 15:16:22.742586 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.742546    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-bundle" (OuterVolumeSpecName: "bundle") pod "20528816-15a2-49af-9dc9-88972c4ca653" (UID: "20528816-15a2-49af-9dc9-88972c4ca653"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:16:22.743949 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.743921    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20528816-15a2-49af-9dc9-88972c4ca653-kube-api-access-5nsdp" (OuterVolumeSpecName: "kube-api-access-5nsdp") pod "20528816-15a2-49af-9dc9-88972c4ca653" (UID: "20528816-15a2-49af-9dc9-88972c4ca653"). InnerVolumeSpecName "kube-api-access-5nsdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:16:22.750423 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.750395    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-util" (OuterVolumeSpecName: "util") pod "20528816-15a2-49af-9dc9-88972c4ca653" (UID: "20528816-15a2-49af-9dc9-88972c4ca653"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:16:22.842761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.842731    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-util\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:16:22.842761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.842762    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nsdp\" (UniqueName: \"kubernetes.io/projected/20528816-15a2-49af-9dc9-88972c4ca653-kube-api-access-5nsdp\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:16:22.842945 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:22.842774    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20528816-15a2-49af-9dc9-88972c4ca653-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:16:23.540516 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:23.540487    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp"
Jul 29 15:16:23.540673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:23.540524    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdbffp" event={"ID":"20528816-15a2-49af-9dc9-88972c4ca653","Type":"ContainerDied","Data":"3af745dc5e6cf9e333f5eb70b26033e424013783030449b451aeb7b5daccbb00"}
Jul 29 15:16:23.540673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:23.540570    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3af745dc5e6cf9e333f5eb70b26033e424013783030449b451aeb7b5daccbb00"
Jul 29 15:16:27.921375 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921333    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"]
Jul 29 15:16:27.921780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921638    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20528816-15a2-49af-9dc9-88972c4ca653" containerName="extract"
Jul 29 15:16:27.921780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921648    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20528816-15a2-49af-9dc9-88972c4ca653" containerName="extract"
Jul 29 15:16:27.921780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921665    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20528816-15a2-49af-9dc9-88972c4ca653" containerName="util"
Jul 29 15:16:27.921780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921672    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20528816-15a2-49af-9dc9-88972c4ca653" containerName="util"
Jul 29 15:16:27.921780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921681    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20528816-15a2-49af-9dc9-88972c4ca653" containerName="pull"
Jul 29 15:16:27.921780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921687    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20528816-15a2-49af-9dc9-88972c4ca653" containerName="pull"
Jul 29 15:16:27.921780 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.921742    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="20528816-15a2-49af-9dc9-88972c4ca653" containerName="extract"
Jul 29 15:16:27.924727 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.924708    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:27.927470 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.927449    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nr6lz\""
Jul 29 15:16:27.928467 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.928433    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 15:16:27.928587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.928434    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:16:27.928587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.928434    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 15:16:27.928587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.928506    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 15:16:27.928587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.928536    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 15:16:27.938290 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:27.938265    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"]
Jul 29 15:16:28.082998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.082961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f3b4bd5-1356-498b-a787-fea41270f146-cert\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.082998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.083007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f3b4bd5-1356-498b-a787-fea41270f146-metrics-cert\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.083283 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.083040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c492\" (UniqueName: \"kubernetes.io/projected/6f3b4bd5-1356-498b-a787-fea41270f146-kube-api-access-8c492\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.083283 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.083133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6f3b4bd5-1356-498b-a787-fea41270f146-manager-config\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.184441 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.184347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6f3b4bd5-1356-498b-a787-fea41270f146-manager-config\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.184441 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.184414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f3b4bd5-1356-498b-a787-fea41270f146-cert\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.184441 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.184431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f3b4bd5-1356-498b-a787-fea41270f146-metrics-cert\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.184720 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.184449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c492\" (UniqueName: \"kubernetes.io/projected/6f3b4bd5-1356-498b-a787-fea41270f146-kube-api-access-8c492\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.185077 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.185052    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6f3b4bd5-1356-498b-a787-fea41270f146-manager-config\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.186958 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.186937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f3b4bd5-1356-498b-a787-fea41270f146-cert\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.187060 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.186970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f3b4bd5-1356-498b-a787-fea41270f146-metrics-cert\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.200476 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.200452    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c492\" (UniqueName: \"kubernetes.io/projected/6f3b4bd5-1356-498b-a787-fea41270f146-kube-api-access-8c492\") pod \"lws-controller-manager-9bd7bffd8-wnp4m\" (UID: \"6f3b4bd5-1356-498b-a787-fea41270f146\") " pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.234165 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.234126    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:28.399373 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.399343    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"]
Jul 29 15:16:28.401865 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:16:28.401842    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f3b4bd5_1356_498b_a787_fea41270f146.slice/crio-5ddc8d84f270af685a56d99aba3e7493d0266a688fab56be1abc1aa3c37b1849 WatchSource:0}: Error finding container 5ddc8d84f270af685a56d99aba3e7493d0266a688fab56be1abc1aa3c37b1849: Status 404 returned error can't find the container with id 5ddc8d84f270af685a56d99aba3e7493d0266a688fab56be1abc1aa3c37b1849
Jul 29 15:16:28.568589 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:28.568498    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m" event={"ID":"6f3b4bd5-1356-498b-a787-fea41270f146","Type":"ContainerStarted","Data":"5ddc8d84f270af685a56d99aba3e7493d0266a688fab56be1abc1aa3c37b1849"}
Jul 29 15:16:30.580127 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:30.580082    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m" event={"ID":"6f3b4bd5-1356-498b-a787-fea41270f146","Type":"ContainerStarted","Data":"204bc9dd6573e5c97feb51e8d919c2a4fdb3d559b9f87809acd021a4023df70b"}
Jul 29 15:16:30.580601 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:30.580218    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:30.600475 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:30.600422    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m" podStartSLOduration=2.049584715 podStartE2EDuration="3.600404064s" totalImagesPullingTime="1.550819349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:16:30.598866139 +0000 UTC m=+328.892802959" watchObservedRunningTime="2026-07-29 15:16:30.600404064 +0000 UTC m=+328.894340866"
Jul 29 15:16:34.387342 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.387302    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"]
Jul 29 15:16:34.391665 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.391639    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.394457 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.394430    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 15:16:34.394605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.394514    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 15:16:34.394605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.394440    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-djg4x\""
Jul 29 15:16:34.394778 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.394761    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 15:16:34.394830 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.394761    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 15:16:34.406073 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.406040    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"]
Jul 29 15:16:34.539890 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.539846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/736f8df5-cf77-4d6b-97d9-403811519e1a-manifests\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.539890 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.539887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/736f8df5-cf77-4d6b-97d9-403811519e1a-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.540110 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.539953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/736f8df5-cf77-4d6b-97d9-403811519e1a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.540110 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.539978    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b94w2\" (UniqueName: \"kubernetes.io/projected/736f8df5-cf77-4d6b-97d9-403811519e1a-kube-api-access-b94w2\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.540110 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.540028    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736f8df5-cf77-4d6b-97d9-403811519e1a-tmp\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.641486 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.641386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/736f8df5-cf77-4d6b-97d9-403811519e1a-manifests\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.641486 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.641426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/736f8df5-cf77-4d6b-97d9-403811519e1a-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.641486 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.641466    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/736f8df5-cf77-4d6b-97d9-403811519e1a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.641771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.641491    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b94w2\" (UniqueName: \"kubernetes.io/projected/736f8df5-cf77-4d6b-97d9-403811519e1a-kube-api-access-b94w2\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.641771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.641654    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736f8df5-cf77-4d6b-97d9-403811519e1a-tmp\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.641878 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.641834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/736f8df5-cf77-4d6b-97d9-403811519e1a-manifests\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.643913 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.643888    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736f8df5-cf77-4d6b-97d9-403811519e1a-tmp\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.644117 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.644098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/736f8df5-cf77-4d6b-97d9-403811519e1a-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.644156 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.644104    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/736f8df5-cf77-4d6b-97d9-403811519e1a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.654473 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.654442    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b94w2\" (UniqueName: \"kubernetes.io/projected/736f8df5-cf77-4d6b-97d9-403811519e1a-kube-api-access-b94w2\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-lpwcm\" (UID: \"736f8df5-cf77-4d6b-97d9-403811519e1a\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.702280 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.702243    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:34.851028 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:34.850995    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"]
Jul 29 15:16:34.855069 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:16:34.855039    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736f8df5_cf77_4d6b_97d9_403811519e1a.slice/crio-4d1f07282bdd8459735c11d61697f145cc38534a29612ebf73e5d767fb7e8f98 WatchSource:0}: Error finding container 4d1f07282bdd8459735c11d61697f145cc38534a29612ebf73e5d767fb7e8f98: Status 404 returned error can't find the container with id 4d1f07282bdd8459735c11d61697f145cc38534a29612ebf73e5d767fb7e8f98
Jul 29 15:16:35.605796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:35.605744    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm" event={"ID":"736f8df5-cf77-4d6b-97d9-403811519e1a","Type":"ContainerStarted","Data":"4d1f07282bdd8459735c11d61697f145cc38534a29612ebf73e5d767fb7e8f98"}
Jul 29 15:16:38.622980 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:38.622941    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm" event={"ID":"736f8df5-cf77-4d6b-97d9-403811519e1a","Type":"ContainerStarted","Data":"781c0c42f4480c7ddd5fc2e8c23119f91c06df9557b0afc17fc96e230962f2df"}
Jul 29 15:16:39.629037 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:39.629004    2560 generic.go:356] "Generic (PLEG): container finished" podID="736f8df5-cf77-4d6b-97d9-403811519e1a" containerID="781c0c42f4480c7ddd5fc2e8c23119f91c06df9557b0afc17fc96e230962f2df" exitCode=0
Jul 29 15:16:39.629437 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:39.629088    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm" event={"ID":"736f8df5-cf77-4d6b-97d9-403811519e1a","Type":"ContainerDied","Data":"781c0c42f4480c7ddd5fc2e8c23119f91c06df9557b0afc17fc96e230962f2df"}
Jul 29 15:16:41.587146 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:41.587119    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-9bd7bffd8-wnp4m"
Jul 29 15:16:42.645718 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:42.645685    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm" event={"ID":"736f8df5-cf77-4d6b-97d9-403811519e1a","Type":"ContainerStarted","Data":"9cc3a11edb73a986a085cc481eedc40ef50236ca7a1b77a72d65d2b8da34f4f4"}
Jul 29 15:16:42.646120 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:42.645811    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:42.669916 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:42.669873    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm" podStartSLOduration=2.157312633 podStartE2EDuration="8.669858294s" totalImagesPullingTime="5.427452156s" totalInitContainerRuntime="1.085093505s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:16:42.66792322 +0000 UTC m=+340.961860021" watchObservedRunningTime="2026-07-29 15:16:42.669858294 +0000 UTC m=+340.963795169"
Jul 29 15:16:53.652915 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:53.652880    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-lpwcm"
Jul 29 15:16:55.899762 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.899725    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-688955555-hbsc7"]
Jul 29 15:16:55.903045 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.903027    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:55.905657 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.905634    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 15:16:55.906275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.906256    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 15:16:55.906388 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.906305    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sxspt\""
Jul 29 15:16:55.906430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.906393    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 15:16:55.908016 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.908001    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 15:16:55.911828 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.911801    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35d05b49-f710-4eb3-9e29-650cb38ec6a1-tls-certs\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:55.911939 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.911844    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s977f\" (UniqueName: \"kubernetes.io/projected/35d05b49-f710-4eb3-9e29-650cb38ec6a1-kube-api-access-s977f\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:55.911939 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.911876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35d05b49-f710-4eb3-9e29-650cb38ec6a1-tmp\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:55.925409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:55.925382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-688955555-hbsc7"]
Jul 29 15:16:56.013009 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.012976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35d05b49-f710-4eb3-9e29-650cb38ec6a1-tls-certs\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:56.013234 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.013022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s977f\" (UniqueName: \"kubernetes.io/projected/35d05b49-f710-4eb3-9e29-650cb38ec6a1-kube-api-access-s977f\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:56.013234 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.013045    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35d05b49-f710-4eb3-9e29-650cb38ec6a1-tmp\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:56.015696 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.015658    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35d05b49-f710-4eb3-9e29-650cb38ec6a1-tls-certs\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:56.015876 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.015858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35d05b49-f710-4eb3-9e29-650cb38ec6a1-tmp\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:56.029064 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.029023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s977f\" (UniqueName: \"kubernetes.io/projected/35d05b49-f710-4eb3-9e29-650cb38ec6a1-kube-api-access-s977f\") pod \"kube-auth-proxy-688955555-hbsc7\" (UID: \"35d05b49-f710-4eb3-9e29-650cb38ec6a1\") " pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:56.213154 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.213059    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7"
Jul 29 15:16:56.363230 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.363204    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-688955555-hbsc7"]
Jul 29 15:16:56.365791 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:16:56.365761    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d05b49_f710_4eb3_9e29_650cb38ec6a1.slice/crio-be92edcf03d313928e9936827c9970fd4ea989405be340761694d7f957892092 WatchSource:0}: Error finding container be92edcf03d313928e9936827c9970fd4ea989405be340761694d7f957892092: Status 404 returned error can't find the container with id be92edcf03d313928e9936827c9970fd4ea989405be340761694d7f957892092
Jul 29 15:16:56.711239 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:56.711171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7" event={"ID":"35d05b49-f710-4eb3-9e29-650cb38ec6a1","Type":"ContainerStarted","Data":"be92edcf03d313928e9936827c9970fd4ea989405be340761694d7f957892092"}
Jul 29 15:16:59.728553 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:59.728516    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7" event={"ID":"35d05b49-f710-4eb3-9e29-650cb38ec6a1","Type":"ContainerStarted","Data":"55f73d7ea05300a96b07f124da857b86b84d0aa9450b8875a25f87405471eabb"}
Jul 29 15:16:59.750802 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:16:59.750752    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-688955555-hbsc7" podStartSLOduration=1.7774754499999998 podStartE2EDuration="4.750735936s" totalImagesPullingTime="2.973260486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:16:59.74956373 +0000 UTC m=+358.043500532" watchObservedRunningTime="2026-07-29 15:16:59.750735936 +0000 UTC m=+358.044672737"
Jul 29 15:18:12.262948 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.262917    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"]
Jul 29 15:18:12.266300 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.266285    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.268788 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.268757    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wt4gx\""
Jul 29 15:18:12.268919 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.268870    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 15:18:12.269398 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.269382    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 15:18:12.275285 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.275257    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"]
Jul 29 15:18:12.346488 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.346453    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5ndt\" (UniqueName: \"kubernetes.io/projected/5bb2d649-1503-4f11-9513-a54f19a896a6-kube-api-access-g5ndt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.346488 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.346500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.346732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.346564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.446954 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.446923    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.447142 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.446995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5ndt\" (UniqueName: \"kubernetes.io/projected/5bb2d649-1503-4f11-9513-a54f19a896a6-kube-api-access-g5ndt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.447142 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.447023    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.447400 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.447379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.447400 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.447393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.456615 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.456592    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5ndt\" (UniqueName: \"kubernetes.io/projected/5bb2d649-1503-4f11-9513-a54f19a896a6-kube-api-access-g5ndt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.576770 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.576684    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:12.656565 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.656531    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"]
Jul 29 15:18:12.660586 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.660561    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.668208 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.668171    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"]
Jul 29 15:18:12.714925 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.714763    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"]
Jul 29 15:18:12.717517 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:18:12.717490    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb2d649_1503_4f11_9513_a54f19a896a6.slice/crio-58d2f9accdc4192945956c682ccf87a6c71678aae584be38cdd90bcb2789fc9b WatchSource:0}: Error finding container 58d2f9accdc4192945956c682ccf87a6c71678aae584be38cdd90bcb2789fc9b: Status 404 returned error can't find the container with id 58d2f9accdc4192945956c682ccf87a6c71678aae584be38cdd90bcb2789fc9b
Jul 29 15:18:12.750025 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.749987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.750025 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.750023    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.750236 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.750042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8sdn\" (UniqueName: \"kubernetes.io/projected/e97debba-fa28-4455-9582-e282c56a4ac0-kube-api-access-h8sdn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.850749 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.850714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.850906 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.850754    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.850906 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.850774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8sdn\" (UniqueName: \"kubernetes.io/projected/e97debba-fa28-4455-9582-e282c56a4ac0-kube-api-access-h8sdn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.851100 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.851082    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.851140 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.851129    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.860648 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.860620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8sdn\" (UniqueName: \"kubernetes.io/projected/e97debba-fa28-4455-9582-e282c56a4ac0-kube-api-access-h8sdn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:12.973914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:12.973879    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:13.065286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.065250    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"]
Jul 29 15:18:13.068983 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.068962    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.076483 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.074808    2560 generic.go:356] "Generic (PLEG): container finished" podID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerID="f4136c79daabdfd8fda573265fc11c4c3668fa5f9a5ad267e4f67141b3ba8791" exitCode=0
Jul 29 15:18:13.076483 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.074877    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5" event={"ID":"5bb2d649-1503-4f11-9513-a54f19a896a6","Type":"ContainerDied","Data":"f4136c79daabdfd8fda573265fc11c4c3668fa5f9a5ad267e4f67141b3ba8791"}
Jul 29 15:18:13.076483 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.074919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5" event={"ID":"5bb2d649-1503-4f11-9513-a54f19a896a6","Type":"ContainerStarted","Data":"58d2f9accdc4192945956c682ccf87a6c71678aae584be38cdd90bcb2789fc9b"}
Jul 29 15:18:13.079235 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.079209    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"]
Jul 29 15:18:13.115348 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.115323    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"]
Jul 29 15:18:13.117536 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:18:13.117510    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97debba_fa28_4455_9582_e282c56a4ac0.slice/crio-5389d6d30c463aff65f11ebf1e3e8f3eb2d943466df1ec8f54a6a112e3d40bc8 WatchSource:0}: Error finding container 5389d6d30c463aff65f11ebf1e3e8f3eb2d943466df1ec8f54a6a112e3d40bc8: Status 404 returned error can't find the container with id 5389d6d30c463aff65f11ebf1e3e8f3eb2d943466df1ec8f54a6a112e3d40bc8
Jul 29 15:18:13.153346 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.153318    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.153470 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.153358    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rttbs\" (UniqueName: \"kubernetes.io/projected/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-kube-api-access-rttbs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.153544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.153476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.254743 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.254699    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.254743 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.254744    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rttbs\" (UniqueName: \"kubernetes.io/projected/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-kube-api-access-rttbs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.255176 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.254789    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.255176 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.255145    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.255176 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.255141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.268105 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.268079    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rttbs\" (UniqueName: \"kubernetes.io/projected/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-kube-api-access-rttbs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.382848 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.382766    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:13.465036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.464362    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"]
Jul 29 15:18:13.468962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.468936    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.475765 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.475704    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"]
Jul 29 15:18:13.552861 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.552827    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"]
Jul 29 15:18:13.556270 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:18:13.556221    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547b007a_f31c_4dd2_bcc8_1bdddfbc6007.slice/crio-d40f85f059b7b69539768deadc270f2be90c5f0a22cfdb51346d3db231dc97a8 WatchSource:0}: Error finding container d40f85f059b7b69539768deadc270f2be90c5f0a22cfdb51346d3db231dc97a8: Status 404 returned error can't find the container with id d40f85f059b7b69539768deadc270f2be90c5f0a22cfdb51346d3db231dc97a8
Jul 29 15:18:13.557217 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.557177    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss6w2\" (UniqueName: \"kubernetes.io/projected/90e935cf-9678-45d1-a058-ffa367641195-kube-api-access-ss6w2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.557335 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.557293    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.557397 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.557362    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.658626 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.658594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.658774 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.658679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss6w2\" (UniqueName: \"kubernetes.io/projected/90e935cf-9678-45d1-a058-ffa367641195-kube-api-access-ss6w2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.658774 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.658730    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.658984 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.658932    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.659061 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.659037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.671896 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.671872    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss6w2\" (UniqueName: \"kubernetes.io/projected/90e935cf-9678-45d1-a058-ffa367641195-kube-api-access-ss6w2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.784701 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.784666    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:13.940717 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:13.940690    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"]
Jul 29 15:18:13.943484 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:18:13.943459    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e935cf_9678_45d1_a058_ffa367641195.slice/crio-bb04c0de73cf71f42bf6692dbcf2e777fcd7499c5bb79f8aaaece7ba383c5e5f WatchSource:0}: Error finding container bb04c0de73cf71f42bf6692dbcf2e777fcd7499c5bb79f8aaaece7ba383c5e5f: Status 404 returned error can't find the container with id bb04c0de73cf71f42bf6692dbcf2e777fcd7499c5bb79f8aaaece7ba383c5e5f
Jul 29 15:18:14.081384 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.081345    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6" event={"ID":"90e935cf-9678-45d1-a058-ffa367641195","Type":"ContainerStarted","Data":"b9f704f7829192bddb477b068726dae6cc11d47235021911535ce9d9302ef7e9"}
Jul 29 15:18:14.081531 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.081402    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6" event={"ID":"90e935cf-9678-45d1-a058-ffa367641195","Type":"ContainerStarted","Data":"bb04c0de73cf71f42bf6692dbcf2e777fcd7499c5bb79f8aaaece7ba383c5e5f"}
Jul 29 15:18:14.083240 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.083216    2560 generic.go:356] "Generic (PLEG): container finished" podID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerID="69fb3b1742f28214e9987ea9209d68d1f6dc34a31c2de9b35e8371b5142f3ce9" exitCode=0
Jul 29 15:18:14.083380 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.083353    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5" event={"ID":"5bb2d649-1503-4f11-9513-a54f19a896a6","Type":"ContainerDied","Data":"69fb3b1742f28214e9987ea9209d68d1f6dc34a31c2de9b35e8371b5142f3ce9"}
Jul 29 15:18:14.084910 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.084884    2560 generic.go:356] "Generic (PLEG): container finished" podID="e97debba-fa28-4455-9582-e282c56a4ac0" containerID="134e777cecd9f88927ee0d0354b5297801e0a31635bcda9b2286fb43ec075ad7" exitCode=0
Jul 29 15:18:14.084990 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.084963    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt" event={"ID":"e97debba-fa28-4455-9582-e282c56a4ac0","Type":"ContainerDied","Data":"134e777cecd9f88927ee0d0354b5297801e0a31635bcda9b2286fb43ec075ad7"}
Jul 29 15:18:14.085054 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.085009    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt" event={"ID":"e97debba-fa28-4455-9582-e282c56a4ac0","Type":"ContainerStarted","Data":"5389d6d30c463aff65f11ebf1e3e8f3eb2d943466df1ec8f54a6a112e3d40bc8"}
Jul 29 15:18:14.086500 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.086477    2560 generic.go:356] "Generic (PLEG): container finished" podID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerID="66901fdc3b15c8fc466aa42191dee2154e467391f3559e574d006871e82e5cc8" exitCode=0
Jul 29 15:18:14.086587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.086524    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx" event={"ID":"547b007a-f31c-4dd2-bcc8-1bdddfbc6007","Type":"ContainerDied","Data":"66901fdc3b15c8fc466aa42191dee2154e467391f3559e574d006871e82e5cc8"}
Jul 29 15:18:14.086587 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:14.086558    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx" event={"ID":"547b007a-f31c-4dd2-bcc8-1bdddfbc6007","Type":"ContainerStarted","Data":"d40f85f059b7b69539768deadc270f2be90c5f0a22cfdb51346d3db231dc97a8"}
Jul 29 15:18:15.094281 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:15.094249    2560 generic.go:356] "Generic (PLEG): container finished" podID="90e935cf-9678-45d1-a058-ffa367641195" containerID="b9f704f7829192bddb477b068726dae6cc11d47235021911535ce9d9302ef7e9" exitCode=0
Jul 29 15:18:15.094621 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:15.094306    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6" event={"ID":"90e935cf-9678-45d1-a058-ffa367641195","Type":"ContainerDied","Data":"b9f704f7829192bddb477b068726dae6cc11d47235021911535ce9d9302ef7e9"}
Jul 29 15:18:15.097030 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:15.097008    2560 generic.go:356] "Generic (PLEG): container finished" podID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerID="2214e3f3f57a1e345f21905f1cf260157360b2278d01920bdd00ef6cb9393ad0" exitCode=0
Jul 29 15:18:15.097136 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:15.097054    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5" event={"ID":"5bb2d649-1503-4f11-9513-a54f19a896a6","Type":"ContainerDied","Data":"2214e3f3f57a1e345f21905f1cf260157360b2278d01920bdd00ef6cb9393ad0"}
Jul 29 15:18:16.109285 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.109248    2560 generic.go:356] "Generic (PLEG): container finished" podID="90e935cf-9678-45d1-a058-ffa367641195" containerID="ded82ec185c30b8b951641977e028c848440cb7ca975e1d3ee2cd9a6a8a2a34a" exitCode=0
Jul 29 15:18:16.109689 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.109291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6" event={"ID":"90e935cf-9678-45d1-a058-ffa367641195","Type":"ContainerDied","Data":"ded82ec185c30b8b951641977e028c848440cb7ca975e1d3ee2cd9a6a8a2a34a"}
Jul 29 15:18:16.111233 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.111212    2560 generic.go:356] "Generic (PLEG): container finished" podID="e97debba-fa28-4455-9582-e282c56a4ac0" containerID="20ae28f5b8ca43c5e52bc69fa8374c4f96e3edc0f4e902102f0484c7b1d2f4f8" exitCode=0
Jul 29 15:18:16.111327 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.111288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt" event={"ID":"e97debba-fa28-4455-9582-e282c56a4ac0","Type":"ContainerDied","Data":"20ae28f5b8ca43c5e52bc69fa8374c4f96e3edc0f4e902102f0484c7b1d2f4f8"}
Jul 29 15:18:16.113104 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.113088    2560 generic.go:356] "Generic (PLEG): container finished" podID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerID="af39fcd562e2374f07b27642f76de718ec2deced3059232491f386030d14932f" exitCode=0
Jul 29 15:18:16.113240 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.113149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx" event={"ID":"547b007a-f31c-4dd2-bcc8-1bdddfbc6007","Type":"ContainerDied","Data":"af39fcd562e2374f07b27642f76de718ec2deced3059232491f386030d14932f"}
Jul 29 15:18:16.266838 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.266809    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:16.381385 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.381300    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5ndt\" (UniqueName: \"kubernetes.io/projected/5bb2d649-1503-4f11-9513-a54f19a896a6-kube-api-access-g5ndt\") pod \"5bb2d649-1503-4f11-9513-a54f19a896a6\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") "
Jul 29 15:18:16.381532 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.381400    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-bundle\") pod \"5bb2d649-1503-4f11-9513-a54f19a896a6\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") "
Jul 29 15:18:16.381532 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.381441    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-util\") pod \"5bb2d649-1503-4f11-9513-a54f19a896a6\" (UID: \"5bb2d649-1503-4f11-9513-a54f19a896a6\") "
Jul 29 15:18:16.381914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.381886    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-bundle" (OuterVolumeSpecName: "bundle") pod "5bb2d649-1503-4f11-9513-a54f19a896a6" (UID: "5bb2d649-1503-4f11-9513-a54f19a896a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:16.383639 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.383613    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bb2d649-1503-4f11-9513-a54f19a896a6-kube-api-access-g5ndt" (OuterVolumeSpecName: "kube-api-access-g5ndt") pod "5bb2d649-1503-4f11-9513-a54f19a896a6" (UID: "5bb2d649-1503-4f11-9513-a54f19a896a6"). InnerVolumeSpecName "kube-api-access-g5ndt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:18:16.387393 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.387348    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-util" (OuterVolumeSpecName: "util") pod "5bb2d649-1503-4f11-9513-a54f19a896a6" (UID: "5bb2d649-1503-4f11-9513-a54f19a896a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:16.482296 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.482263    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:16.482296 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.482292    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bb2d649-1503-4f11-9513-a54f19a896a6-util\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:16.482296 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:16.482303    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5ndt\" (UniqueName: \"kubernetes.io/projected/5bb2d649-1503-4f11-9513-a54f19a896a6-kube-api-access-g5ndt\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:17.120542 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.120455    2560 generic.go:356] "Generic (PLEG): container finished" podID="e97debba-fa28-4455-9582-e282c56a4ac0" containerID="1dc9766851a43d2d2faafd0622b8fc5b50979712af32675fd1317bf4892ce150" exitCode=0
Jul 29 15:18:17.120983 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.120542    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt" event={"ID":"e97debba-fa28-4455-9582-e282c56a4ac0","Type":"ContainerDied","Data":"1dc9766851a43d2d2faafd0622b8fc5b50979712af32675fd1317bf4892ce150"}
Jul 29 15:18:17.122570 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.122548    2560 generic.go:356] "Generic (PLEG): container finished" podID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerID="48668a5c92eb4fb8751a7348a1d64ca47a6e341ad65c774c7b9c20d617ea8473" exitCode=0
Jul 29 15:18:17.122675 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.122634    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx" event={"ID":"547b007a-f31c-4dd2-bcc8-1bdddfbc6007","Type":"ContainerDied","Data":"48668a5c92eb4fb8751a7348a1d64ca47a6e341ad65c774c7b9c20d617ea8473"}
Jul 29 15:18:17.124578 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.124521    2560 generic.go:356] "Generic (PLEG): container finished" podID="90e935cf-9678-45d1-a058-ffa367641195" containerID="1928e4afc9b11a167442ccc60ecd445224a8b9e734839946ca9a5795ed0002b5" exitCode=0
Jul 29 15:18:17.124663 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.124601    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6" event={"ID":"90e935cf-9678-45d1-a058-ffa367641195","Type":"ContainerDied","Data":"1928e4afc9b11a167442ccc60ecd445224a8b9e734839946ca9a5795ed0002b5"}
Jul 29 15:18:17.126379 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.126354    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5"
Jul 29 15:18:17.126379 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.126366    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5" event={"ID":"5bb2d649-1503-4f11-9513-a54f19a896a6","Type":"ContainerDied","Data":"58d2f9accdc4192945956c682ccf87a6c71678aae584be38cdd90bcb2789fc9b"}
Jul 29 15:18:17.126501 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:17.126401    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58d2f9accdc4192945956c682ccf87a6c71678aae584be38cdd90bcb2789fc9b"
Jul 29 15:18:18.271533 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.271506    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:18.322415 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.322392    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:18.325335 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.325315    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:18.399354 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399259    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-bundle\") pod \"e97debba-fa28-4455-9582-e282c56a4ac0\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") "
Jul 29 15:18:18.399354 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399300    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rttbs\" (UniqueName: \"kubernetes.io/projected/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-kube-api-access-rttbs\") pod \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") "
Jul 29 15:18:18.399354 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399320    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-bundle\") pod \"90e935cf-9678-45d1-a058-ffa367641195\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") "
Jul 29 15:18:18.399624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399358    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-util\") pod \"e97debba-fa28-4455-9582-e282c56a4ac0\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") "
Jul 29 15:18:18.399624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399382    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss6w2\" (UniqueName: \"kubernetes.io/projected/90e935cf-9678-45d1-a058-ffa367641195-kube-api-access-ss6w2\") pod \"90e935cf-9678-45d1-a058-ffa367641195\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") "
Jul 29 15:18:18.399624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399419    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-bundle\") pod \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") "
Jul 29 15:18:18.399624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399469    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-util\") pod \"90e935cf-9678-45d1-a058-ffa367641195\" (UID: \"90e935cf-9678-45d1-a058-ffa367641195\") "
Jul 29 15:18:18.399624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399520    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-util\") pod \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\" (UID: \"547b007a-f31c-4dd2-bcc8-1bdddfbc6007\") "
Jul 29 15:18:18.399624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.399551    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8sdn\" (UniqueName: \"kubernetes.io/projected/e97debba-fa28-4455-9582-e282c56a4ac0-kube-api-access-h8sdn\") pod \"e97debba-fa28-4455-9582-e282c56a4ac0\" (UID: \"e97debba-fa28-4455-9582-e282c56a4ac0\") "
Jul 29 15:18:18.400452 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.400030    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-bundle" (OuterVolumeSpecName: "bundle") pod "e97debba-fa28-4455-9582-e282c56a4ac0" (UID: "e97debba-fa28-4455-9582-e282c56a4ac0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:18.400452 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.400126    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-bundle" (OuterVolumeSpecName: "bundle") pod "90e935cf-9678-45d1-a058-ffa367641195" (UID: "90e935cf-9678-45d1-a058-ffa367641195"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:18.400452 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.400399    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-bundle" (OuterVolumeSpecName: "bundle") pod "547b007a-f31c-4dd2-bcc8-1bdddfbc6007" (UID: "547b007a-f31c-4dd2-bcc8-1bdddfbc6007"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:18.402088 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.402051    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e935cf-9678-45d1-a058-ffa367641195-kube-api-access-ss6w2" (OuterVolumeSpecName: "kube-api-access-ss6w2") pod "90e935cf-9678-45d1-a058-ffa367641195" (UID: "90e935cf-9678-45d1-a058-ffa367641195"). InnerVolumeSpecName "kube-api-access-ss6w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:18:18.402220 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.402127    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e97debba-fa28-4455-9582-e282c56a4ac0-kube-api-access-h8sdn" (OuterVolumeSpecName: "kube-api-access-h8sdn") pod "e97debba-fa28-4455-9582-e282c56a4ac0" (UID: "e97debba-fa28-4455-9582-e282c56a4ac0"). InnerVolumeSpecName "kube-api-access-h8sdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:18:18.402220 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.402134    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-kube-api-access-rttbs" (OuterVolumeSpecName: "kube-api-access-rttbs") pod "547b007a-f31c-4dd2-bcc8-1bdddfbc6007" (UID: "547b007a-f31c-4dd2-bcc8-1bdddfbc6007"). InnerVolumeSpecName "kube-api-access-rttbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:18:18.406036 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.406009    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-util" (OuterVolumeSpecName: "util") pod "90e935cf-9678-45d1-a058-ffa367641195" (UID: "90e935cf-9678-45d1-a058-ffa367641195"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:18.406226 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.406200    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-util" (OuterVolumeSpecName: "util") pod "e97debba-fa28-4455-9582-e282c56a4ac0" (UID: "e97debba-fa28-4455-9582-e282c56a4ac0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:18.406421 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.406400    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-util" (OuterVolumeSpecName: "util") pod "547b007a-f31c-4dd2-bcc8-1bdddfbc6007" (UID: "547b007a-f31c-4dd2-bcc8-1bdddfbc6007"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:18.500999 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.500961    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8sdn\" (UniqueName: \"kubernetes.io/projected/e97debba-fa28-4455-9582-e282c56a4ac0-kube-api-access-h8sdn\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.500999 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.500991    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.500999 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.501001    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rttbs\" (UniqueName: \"kubernetes.io/projected/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-kube-api-access-rttbs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.500999 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.501013    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.501326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.501021    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e97debba-fa28-4455-9582-e282c56a4ac0-util\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.501326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.501029    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss6w2\" (UniqueName: \"kubernetes.io/projected/90e935cf-9678-45d1-a058-ffa367641195-kube-api-access-ss6w2\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.501326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.501037    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.501326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.501046    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90e935cf-9678-45d1-a058-ffa367641195-util\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:18.501326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:18.501052    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/547b007a-f31c-4dd2-bcc8-1bdddfbc6007-util\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:19.137881 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.137845    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx"
Jul 29 15:18:19.138122 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.137848    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx" event={"ID":"547b007a-f31c-4dd2-bcc8-1bdddfbc6007","Type":"ContainerDied","Data":"d40f85f059b7b69539768deadc270f2be90c5f0a22cfdb51346d3db231dc97a8"}
Jul 29 15:18:19.138122 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.137976    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d40f85f059b7b69539768deadc270f2be90c5f0a22cfdb51346d3db231dc97a8"
Jul 29 15:18:19.139815 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.139794    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6"
Jul 29 15:18:19.139924 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.139819    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6" event={"ID":"90e935cf-9678-45d1-a058-ffa367641195","Type":"ContainerDied","Data":"bb04c0de73cf71f42bf6692dbcf2e777fcd7499c5bb79f8aaaece7ba383c5e5f"}
Jul 29 15:18:19.139924 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.139866    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb04c0de73cf71f42bf6692dbcf2e777fcd7499c5bb79f8aaaece7ba383c5e5f"
Jul 29 15:18:19.141718 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.141695    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt" event={"ID":"e97debba-fa28-4455-9582-e282c56a4ac0","Type":"ContainerDied","Data":"5389d6d30c463aff65f11ebf1e3e8f3eb2d943466df1ec8f54a6a112e3d40bc8"}
Jul 29 15:18:19.141718 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.141712    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt"
Jul 29 15:18:19.141862 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:19.141735    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5389d6d30c463aff65f11ebf1e3e8f3eb2d943466df1ec8f54a6a112e3d40bc8"
Jul 29 15:18:21.740466 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740423    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-857fdfbb7f-bbhqn"]
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740887    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerName="extract"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740907    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerName="extract"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740922    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerName="extract"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740930    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerName="extract"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740938    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e97debba-fa28-4455-9582-e282c56a4ac0" containerName="extract"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740947    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e97debba-fa28-4455-9582-e282c56a4ac0" containerName="extract"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740962    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e935cf-9678-45d1-a058-ffa367641195" containerName="util"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740970    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e935cf-9678-45d1-a058-ffa367641195" containerName="util"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740980    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e97debba-fa28-4455-9582-e282c56a4ac0" containerName="pull"
Jul 29 15:18:21.740998 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.740987    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e97debba-fa28-4455-9582-e282c56a4ac0" containerName="pull"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741006    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerName="util"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741017    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerName="util"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741027    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e97debba-fa28-4455-9582-e282c56a4ac0" containerName="util"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741036    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e97debba-fa28-4455-9582-e282c56a4ac0" containerName="util"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741045    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerName="util"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741053    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerName="util"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741064    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e935cf-9678-45d1-a058-ffa367641195" containerName="extract"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741072    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e935cf-9678-45d1-a058-ffa367641195" containerName="extract"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741082    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e935cf-9678-45d1-a058-ffa367641195" containerName="pull"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741090    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e935cf-9678-45d1-a058-ffa367641195" containerName="pull"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741103    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerName="pull"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741111    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerName="pull"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741150    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerName="pull"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741159    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerName="pull"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741264    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="547b007a-f31c-4dd2-bcc8-1bdddfbc6007" containerName="extract"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741276    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e97debba-fa28-4455-9582-e282c56a4ac0" containerName="extract"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741291    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bb2d649-1503-4f11-9513-a54f19a896a6" containerName="extract"
Jul 29 15:18:21.741575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.741301    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="90e935cf-9678-45d1-a058-ffa367641195" containerName="extract"
Jul 29 15:18:21.745787 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.745766    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.755692 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.755669    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857fdfbb7f-bbhqn"]
Jul 29 15:18:21.828257 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.828225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-oauth-config\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.828401 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.828272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4mdh\" (UniqueName: \"kubernetes.io/projected/705f19d6-9d73-45b3-bedd-d981f460bbcc-kube-api-access-n4mdh\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.828401 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.828353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-service-ca\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.828472 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.828406    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-serving-cert\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.828472 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.828425    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-trusted-ca-bundle\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.828539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.828474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-config\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.828539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.828520    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-oauth-serving-cert\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.929534 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.929483    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-service-ca\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.929534 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.929545    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-serving-cert\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.929761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.929560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-trusted-ca-bundle\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.929761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.929582    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-config\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.929761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.929613    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-oauth-serving-cert\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.929761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.929635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-oauth-config\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.929761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.929668    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4mdh\" (UniqueName: \"kubernetes.io/projected/705f19d6-9d73-45b3-bedd-d981f460bbcc-kube-api-access-n4mdh\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.930386 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.930359    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-service-ca\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.930508 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.930467    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-config\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.930573 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.930503    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-oauth-serving-cert\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.930573 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.930503    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705f19d6-9d73-45b3-bedd-d981f460bbcc-trusted-ca-bundle\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.932144 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.932115    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-oauth-config\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.932275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.932218    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/705f19d6-9d73-45b3-bedd-d981f460bbcc-console-serving-cert\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:21.940426 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:21.940398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4mdh\" (UniqueName: \"kubernetes.io/projected/705f19d6-9d73-45b3-bedd-d981f460bbcc-kube-api-access-n4mdh\") pod \"console-857fdfbb7f-bbhqn\" (UID: \"705f19d6-9d73-45b3-bedd-d981f460bbcc\") " pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:22.055879 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:22.055784    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:22.198974 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:22.198948    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857fdfbb7f-bbhqn"]
Jul 29 15:18:22.201196 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:18:22.201151    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705f19d6_9d73_45b3_bedd_d981f460bbcc.slice/crio-0584c1249b775e852b8c6f78aa4ec14346bbef719aa5ef265d52e68d4fdf1c60 WatchSource:0}: Error finding container 0584c1249b775e852b8c6f78aa4ec14346bbef719aa5ef265d52e68d4fdf1c60: Status 404 returned error can't find the container with id 0584c1249b775e852b8c6f78aa4ec14346bbef719aa5ef265d52e68d4fdf1c60
Jul 29 15:18:23.164687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:23.164652    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857fdfbb7f-bbhqn" event={"ID":"705f19d6-9d73-45b3-bedd-d981f460bbcc","Type":"ContainerStarted","Data":"2c30b91d270821495a39d47c327eebfd4d7e2f39f1482c73a1385e167976387d"}
Jul 29 15:18:23.165066 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:23.164706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857fdfbb7f-bbhqn" event={"ID":"705f19d6-9d73-45b3-bedd-d981f460bbcc","Type":"ContainerStarted","Data":"0584c1249b775e852b8c6f78aa4ec14346bbef719aa5ef265d52e68d4fdf1c60"}
Jul 29 15:18:23.188778 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:23.188725    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-857fdfbb7f-bbhqn" podStartSLOduration=2.188709408 podStartE2EDuration="2.188709408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:23.187066602 +0000 UTC m=+441.481003417" watchObservedRunningTime="2026-07-29 15:18:23.188709408 +0000 UTC m=+441.482646211"
Jul 29 15:18:32.057030 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:32.056917    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:32.057030 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:32.056987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:32.061618 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:32.061592    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:32.211230 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:32.211198    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-857fdfbb7f-bbhqn"
Jul 29 15:18:32.262611 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:32.262577    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5959f77cdf-9wxcv"]
Jul 29 15:18:39.302671 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.302631    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"]
Jul 29 15:18:39.306122 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.306098    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"
Jul 29 15:18:39.308698 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.308671    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 15:18:39.308828 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.308671    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 29 15:18:39.309387 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.309370    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 15:18:39.309489 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.309447    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-t9z58\""
Jul 29 15:18:39.317143 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.317122    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"]
Jul 29 15:18:39.490953 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.490915    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npvw2\" (UniqueName: \"kubernetes.io/projected/29753784-4887-4355-8a7e-eff29ab18e9b-kube-api-access-npvw2\") pod \"dns-operator-controller-manager-864dd78c4c-s4lsc\" (UID: \"29753784-4887-4355-8a7e-eff29ab18e9b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"
Jul 29 15:18:39.592019 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.591992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npvw2\" (UniqueName: \"kubernetes.io/projected/29753784-4887-4355-8a7e-eff29ab18e9b-kube-api-access-npvw2\") pod \"dns-operator-controller-manager-864dd78c4c-s4lsc\" (UID: \"29753784-4887-4355-8a7e-eff29ab18e9b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"
Jul 29 15:18:39.602058 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.602024    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npvw2\" (UniqueName: \"kubernetes.io/projected/29753784-4887-4355-8a7e-eff29ab18e9b-kube-api-access-npvw2\") pod \"dns-operator-controller-manager-864dd78c4c-s4lsc\" (UID: \"29753784-4887-4355-8a7e-eff29ab18e9b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"
Jul 29 15:18:39.618886 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.618852    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"
Jul 29 15:18:39.756253 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:39.756226    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"]
Jul 29 15:18:39.758076 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:18:39.758046    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29753784_4887_4355_8a7e_eff29ab18e9b.slice/crio-e48b248fc0462d7c49cfd06ff13c6553025743f493631eddd98f9299f218a7d3 WatchSource:0}: Error finding container e48b248fc0462d7c49cfd06ff13c6553025743f493631eddd98f9299f218a7d3: Status 404 returned error can't find the container with id e48b248fc0462d7c49cfd06ff13c6553025743f493631eddd98f9299f218a7d3
Jul 29 15:18:40.247466 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:40.247429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc" event={"ID":"29753784-4887-4355-8a7e-eff29ab18e9b","Type":"ContainerStarted","Data":"e48b248fc0462d7c49cfd06ff13c6553025743f493631eddd98f9299f218a7d3"}
Jul 29 15:18:43.268498 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:43.268456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc" event={"ID":"29753784-4887-4355-8a7e-eff29ab18e9b","Type":"ContainerStarted","Data":"d4402867080c8ea2c6916de5e95f57ac6fc659d69284e2673d0e5cb52da42fc3"}
Jul 29 15:18:43.268902 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:43.268551    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"
Jul 29 15:18:43.287406 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:43.287346    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc" podStartSLOduration=1.849919546 podStartE2EDuration="4.287329749s" totalImagesPullingTime="2.437410203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:43.286101389 +0000 UTC m=+461.580038195" watchObservedRunningTime="2026-07-29 15:18:43.287329749 +0000 UTC m=+461.581266553"
Jul 29 15:18:52.916337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:52.916300    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"]
Jul 29 15:18:52.921214 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:52.921175    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:52.925229 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:52.925209    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhvk2\""
Jul 29 15:18:52.935034 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:52.935012    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"]
Jul 29 15:18:53.007830 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.007791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4118d7f1-7a25-4acf-b731-26d95e0190dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:53.008021 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.007857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psmk6\" (UniqueName: \"kubernetes.io/projected/4118d7f1-7a25-4acf-b731-26d95e0190dc-kube-api-access-psmk6\") pod \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:53.109177 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.109138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psmk6\" (UniqueName: \"kubernetes.io/projected/4118d7f1-7a25-4acf-b731-26d95e0190dc-kube-api-access-psmk6\") pod \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:53.109390 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.109234    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4118d7f1-7a25-4acf-b731-26d95e0190dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:53.109590 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.109571    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4118d7f1-7a25-4acf-b731-26d95e0190dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:53.119083 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.119046    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psmk6\" (UniqueName: \"kubernetes.io/projected/4118d7f1-7a25-4acf-b731-26d95e0190dc-kube-api-access-psmk6\") pod \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:53.217053 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.216963    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"]
Jul 29 15:18:53.217247 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.217233    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:53.227234 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.227203    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"]
Jul 29 15:18:53.243478 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.243449    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"]
Jul 29 15:18:53.248393 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.248368    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.264460 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.264433    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"]
Jul 29 15:18:53.311290 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.311253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efaadfda-0397-441b-8ec0-94f3fc4767b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-8zp7s\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.311480 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.311327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwvnq\" (UniqueName: \"kubernetes.io/projected/efaadfda-0397-441b-8ec0-94f3fc4767b6-kube-api-access-pwvnq\") pod \"kuadrant-operator-controller-manager-55467d9f7-8zp7s\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.411714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.411678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efaadfda-0397-441b-8ec0-94f3fc4767b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-8zp7s\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.411894 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.411748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvnq\" (UniqueName: \"kubernetes.io/projected/efaadfda-0397-441b-8ec0-94f3fc4767b6-kube-api-access-pwvnq\") pod \"kuadrant-operator-controller-manager-55467d9f7-8zp7s\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.412089 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.412067    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efaadfda-0397-441b-8ec0-94f3fc4767b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-8zp7s\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.421082 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.421055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwvnq\" (UniqueName: \"kubernetes.io/projected/efaadfda-0397-441b-8ec0-94f3fc4767b6-kube-api-access-pwvnq\") pod \"kuadrant-operator-controller-manager-55467d9f7-8zp7s\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.559398 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.559293    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:53.707892 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:53.707867    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"]
Jul 29 15:18:53.709699 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:18:53.709679    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefaadfda_0397_441b_8ec0_94f3fc4767b6.slice/crio-0610ea1b1e3eec1a991da84be16e1a7bc5bdbfbbf8649bc4c14ecf9f7dfebf1e WatchSource:0}: Error finding container 0610ea1b1e3eec1a991da84be16e1a7bc5bdbfbbf8649bc4c14ecf9f7dfebf1e: Status 404 returned error can't find the container with id 0610ea1b1e3eec1a991da84be16e1a7bc5bdbfbbf8649bc4c14ecf9f7dfebf1e
Jul 29 15:18:54.275653 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:54.275629    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s4lsc"
Jul 29 15:18:54.331732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:54.331696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s" event={"ID":"efaadfda-0397-441b-8ec0-94f3fc4767b6","Type":"ContainerStarted","Data":"0610ea1b1e3eec1a991da84be16e1a7bc5bdbfbbf8649bc4c14ecf9f7dfebf1e"}
Jul 29 15:18:55.763797 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:18:55.763758    2560 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-zp6qw_kuadrant-system_4118d7f1-7a25-4acf-b731-26d95e0190dc_0(a1af670f8da18b740c6a5826916f70f7ed9b94513903b4c57eb577c144b27ed6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-zp6qw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 29 15:18:55.764106 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:18:55.763833    2560 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-zp6qw_kuadrant-system_4118d7f1-7a25-4acf-b731-26d95e0190dc_0(a1af670f8da18b740c6a5826916f70f7ed9b94513903b4c57eb577c144b27ed6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-zp6qw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:56.341406 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.341369    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:56.350993 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.350969    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:56.353046 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:18:56.353013    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="4118d7f1-7a25-4acf-b731-26d95e0190dc" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:56.439435 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.439397    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4118d7f1-7a25-4acf-b731-26d95e0190dc-extensions-socket-volume\") pod \"4118d7f1-7a25-4acf-b731-26d95e0190dc\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") "
Jul 29 15:18:56.439616 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.439565    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psmk6\" (UniqueName: \"kubernetes.io/projected/4118d7f1-7a25-4acf-b731-26d95e0190dc-kube-api-access-psmk6\") pod \"4118d7f1-7a25-4acf-b731-26d95e0190dc\" (UID: \"4118d7f1-7a25-4acf-b731-26d95e0190dc\") "
Jul 29 15:18:56.439732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.439702    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4118d7f1-7a25-4acf-b731-26d95e0190dc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4118d7f1-7a25-4acf-b731-26d95e0190dc" (UID: "4118d7f1-7a25-4acf-b731-26d95e0190dc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:18:56.439829 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.439806    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4118d7f1-7a25-4acf-b731-26d95e0190dc-extensions-socket-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:56.442141 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.442112    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4118d7f1-7a25-4acf-b731-26d95e0190dc-kube-api-access-psmk6" (OuterVolumeSpecName: "kube-api-access-psmk6") pod "4118d7f1-7a25-4acf-b731-26d95e0190dc" (UID: "4118d7f1-7a25-4acf-b731-26d95e0190dc"). InnerVolumeSpecName "kube-api-access-psmk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:18:56.540730 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:56.540690    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psmk6\" (UniqueName: \"kubernetes.io/projected/4118d7f1-7a25-4acf-b731-26d95e0190dc-kube-api-access-psmk6\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:57.282883 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:57.282838    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5959f77cdf-9wxcv" podUID="484d3aa4-0026-4593-a538-b234269800ec" containerName="console" containerID="cri-o://df9ad791d3357699888e95a6ffdd4b05e38868900682cf743a42fd0e88f79f34" gracePeriod=15
Jul 29 15:18:57.347229 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:57.347199    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:57.349479 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:18:57.349438    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="4118d7f1-7a25-4acf-b731-26d95e0190dc" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:57.356030 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:18:57.355997    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-zp6qw\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="4118d7f1-7a25-4acf-b731-26d95e0190dc" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zp6qw"
Jul 29 15:18:58.142434 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.142397    2560 patch_prober.go:28] interesting pod/console-5959f77cdf-9wxcv container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused" start-of-body=
Jul 29 15:18:58.142622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.142466    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-5959f77cdf-9wxcv" podUID="484d3aa4-0026-4593-a538-b234269800ec" containerName="console" probeResult="failure" output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused"
Jul 29 15:18:58.167042 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.167001    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4118d7f1-7a25-4acf-b731-26d95e0190dc" path="/var/lib/kubelet/pods/4118d7f1-7a25-4acf-b731-26d95e0190dc/volumes"
Jul 29 15:18:58.354475 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.354448    2560 generic.go:356] "Generic (PLEG): container finished" podID="484d3aa4-0026-4593-a538-b234269800ec" containerID="df9ad791d3357699888e95a6ffdd4b05e38868900682cf743a42fd0e88f79f34" exitCode=0
Jul 29 15:18:58.354763 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.354518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5959f77cdf-9wxcv" event={"ID":"484d3aa4-0026-4593-a538-b234269800ec","Type":"ContainerDied","Data":"df9ad791d3357699888e95a6ffdd4b05e38868900682cf743a42fd0e88f79f34"}
Jul 29 15:18:58.499907 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.499885    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:18:58.557086 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557050    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-serving-cert\") pod \"484d3aa4-0026-4593-a538-b234269800ec\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") "
Jul 29 15:18:58.557264 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557093    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-service-ca\") pod \"484d3aa4-0026-4593-a538-b234269800ec\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") "
Jul 29 15:18:58.557264 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557113    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6748k\" (UniqueName: \"kubernetes.io/projected/484d3aa4-0026-4593-a538-b234269800ec-kube-api-access-6748k\") pod \"484d3aa4-0026-4593-a538-b234269800ec\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") "
Jul 29 15:18:58.557264 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557150    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-trusted-ca-bundle\") pod \"484d3aa4-0026-4593-a538-b234269800ec\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") "
Jul 29 15:18:58.557264 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557204    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-oauth-serving-cert\") pod \"484d3aa4-0026-4593-a538-b234269800ec\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") "
Jul 29 15:18:58.557264 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557235    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-console-config\") pod \"484d3aa4-0026-4593-a538-b234269800ec\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") "
Jul 29 15:18:58.557548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557273    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-oauth-config\") pod \"484d3aa4-0026-4593-a538-b234269800ec\" (UID: \"484d3aa4-0026-4593-a538-b234269800ec\") "
Jul 29 15:18:58.557602 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557547    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-service-ca" (OuterVolumeSpecName: "service-ca") pod "484d3aa4-0026-4593-a538-b234269800ec" (UID: "484d3aa4-0026-4593-a538-b234269800ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:18:58.557602 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557549    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "484d3aa4-0026-4593-a538-b234269800ec" (UID: "484d3aa4-0026-4593-a538-b234269800ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:18:58.557920 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.557894    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "484d3aa4-0026-4593-a538-b234269800ec" (UID: "484d3aa4-0026-4593-a538-b234269800ec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:18:58.558388 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.558366    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-console-config" (OuterVolumeSpecName: "console-config") pod "484d3aa4-0026-4593-a538-b234269800ec" (UID: "484d3aa4-0026-4593-a538-b234269800ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:18:58.559400 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.559385    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "484d3aa4-0026-4593-a538-b234269800ec" (UID: "484d3aa4-0026-4593-a538-b234269800ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:18:58.559684 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.559660    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "484d3aa4-0026-4593-a538-b234269800ec" (UID: "484d3aa4-0026-4593-a538-b234269800ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:18:58.559769 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.559749    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484d3aa4-0026-4593-a538-b234269800ec-kube-api-access-6748k" (OuterVolumeSpecName: "kube-api-access-6748k") pod "484d3aa4-0026-4593-a538-b234269800ec" (UID: "484d3aa4-0026-4593-a538-b234269800ec"). InnerVolumeSpecName "kube-api-access-6748k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:18:58.658914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.658826    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-serving-cert\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:58.658914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.658860    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:58.658914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.658874    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6748k\" (UniqueName: \"kubernetes.io/projected/484d3aa4-0026-4593-a538-b234269800ec-kube-api-access-6748k\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:58.658914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.658887    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-trusted-ca-bundle\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:58.658914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.658898    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-oauth-serving-cert\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:58.658914 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.658911    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484d3aa4-0026-4593-a538-b234269800ec-console-config\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:58.659276 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:58.658923    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484d3aa4-0026-4593-a538-b234269800ec-console-oauth-config\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:18:59.360963 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.360924    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5959f77cdf-9wxcv" event={"ID":"484d3aa4-0026-4593-a538-b234269800ec","Type":"ContainerDied","Data":"871de3f797ad75c661a9daf02fa0b74ab362220d77a993981c7e97a674ff30a5"}
Jul 29 15:18:59.361461 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.360943    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5959f77cdf-9wxcv"
Jul 29 15:18:59.361461 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.360987    2560 scope.go:117] "RemoveContainer" containerID="df9ad791d3357699888e95a6ffdd4b05e38868900682cf743a42fd0e88f79f34"
Jul 29 15:18:59.362946 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.362911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s" event={"ID":"efaadfda-0397-441b-8ec0-94f3fc4767b6","Type":"ContainerStarted","Data":"342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e"}
Jul 29 15:18:59.363056 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.363031    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:18:59.384816 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.384771    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s" podStartSLOduration=1.6893403340000002 podStartE2EDuration="6.384755276s" totalImagesPullingTime="4.695414942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:59.382848763 +0000 UTC m=+477.676785610" watchObservedRunningTime="2026-07-29 15:18:59.384755276 +0000 UTC m=+477.678692078"
Jul 29 15:18:59.396726 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.396698    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5959f77cdf-9wxcv"]
Jul 29 15:18:59.401711 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:18:59.401684    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5959f77cdf-9wxcv"]
Jul 29 15:19:00.166256 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:00.166220    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="484d3aa4-0026-4593-a538-b234269800ec" path="/var/lib/kubelet/pods/484d3aa4-0026-4593-a538-b234269800ec/volumes"
Jul 29 15:19:10.370913 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:10.370879    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:19:23.417770 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.417726    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"]
Jul 29 15:19:23.418326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.417999    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s" podUID="efaadfda-0397-441b-8ec0-94f3fc4767b6" containerName="manager" containerID="cri-o://342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e" gracePeriod=10
Jul 29 15:19:23.667984 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.667927    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:19:23.779356 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.779301    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwvnq\" (UniqueName: \"kubernetes.io/projected/efaadfda-0397-441b-8ec0-94f3fc4767b6-kube-api-access-pwvnq\") pod \"efaadfda-0397-441b-8ec0-94f3fc4767b6\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") "
Jul 29 15:19:23.779548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.779448    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efaadfda-0397-441b-8ec0-94f3fc4767b6-extensions-socket-volume\") pod \"efaadfda-0397-441b-8ec0-94f3fc4767b6\" (UID: \"efaadfda-0397-441b-8ec0-94f3fc4767b6\") "
Jul 29 15:19:23.779799 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.779778    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efaadfda-0397-441b-8ec0-94f3fc4767b6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "efaadfda-0397-441b-8ec0-94f3fc4767b6" (UID: "efaadfda-0397-441b-8ec0-94f3fc4767b6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:19:23.781404 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.781382    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efaadfda-0397-441b-8ec0-94f3fc4767b6-kube-api-access-pwvnq" (OuterVolumeSpecName: "kube-api-access-pwvnq") pod "efaadfda-0397-441b-8ec0-94f3fc4767b6" (UID: "efaadfda-0397-441b-8ec0-94f3fc4767b6"). InnerVolumeSpecName "kube-api-access-pwvnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:19:23.880015 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.879974    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efaadfda-0397-441b-8ec0-94f3fc4767b6-extensions-socket-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:19:23.880015 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:23.880009    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwvnq\" (UniqueName: \"kubernetes.io/projected/efaadfda-0397-441b-8ec0-94f3fc4767b6-kube-api-access-pwvnq\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:19:24.502571 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.502480    2560 generic.go:356] "Generic (PLEG): container finished" podID="efaadfda-0397-441b-8ec0-94f3fc4767b6" containerID="342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e" exitCode=0
Jul 29 15:19:24.502571 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.502530    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"
Jul 29 15:19:24.503025 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.502575    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s" event={"ID":"efaadfda-0397-441b-8ec0-94f3fc4767b6","Type":"ContainerDied","Data":"342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e"}
Jul 29 15:19:24.503025 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.502636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s" event={"ID":"efaadfda-0397-441b-8ec0-94f3fc4767b6","Type":"ContainerDied","Data":"0610ea1b1e3eec1a991da84be16e1a7bc5bdbfbbf8649bc4c14ecf9f7dfebf1e"}
Jul 29 15:19:24.503025 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.502666    2560 scope.go:117] "RemoveContainer" containerID="342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e"
Jul 29 15:19:24.511668 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.511647    2560 scope.go:117] "RemoveContainer" containerID="342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e"
Jul 29 15:19:24.511974 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:19:24.511950    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e\": container with ID starting with 342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e not found: ID does not exist" containerID="342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e"
Jul 29 15:19:24.512031 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.511986    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e"} err="failed to get container status \"342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e\": rpc error: code = NotFound desc = could not find container \"342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e\": container with ID starting with 342caced464c7b084382910ec5e464531db9bbb229152c86ebb574c33b7f211e not found: ID does not exist"
Jul 29 15:19:24.532349 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.532313    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"]
Jul 29 15:19:24.539255 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:24.539222    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8zp7s"]
Jul 29 15:19:26.165416 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:26.165381    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efaadfda-0397-441b-8ec0-94f3fc4767b6" path="/var/lib/kubelet/pods/efaadfda-0397-441b-8ec0-94f3fc4767b6/volumes"
Jul 29 15:19:40.752511 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.752472    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:19:40.752962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.752837    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efaadfda-0397-441b-8ec0-94f3fc4767b6" containerName="manager"
Jul 29 15:19:40.752962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.752852    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="efaadfda-0397-441b-8ec0-94f3fc4767b6" containerName="manager"
Jul 29 15:19:40.752962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.752872    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484d3aa4-0026-4593-a538-b234269800ec" containerName="console"
Jul 29 15:19:40.752962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.752879    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="484d3aa4-0026-4593-a538-b234269800ec" containerName="console"
Jul 29 15:19:40.752962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.752950    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="484d3aa4-0026-4593-a538-b234269800ec" containerName="console"
Jul 29 15:19:40.752962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.752959    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="efaadfda-0397-441b-8ec0-94f3fc4767b6" containerName="manager"
Jul 29 15:19:40.757656 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.757639    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:40.759777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.759750    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 15:19:40.759777 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.759769    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wt4gx\""
Jul 29 15:19:40.767337 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.767307    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:19:40.799168 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.799128    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:19:40.822399 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.822367    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ecda580b-18a1-435d-bb7a-44f97d6afec4-config-file\") pod \"limitador-limitador-54d6d988d5-ggt7p\" (UID: \"ecda580b-18a1-435d-bb7a-44f97d6afec4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:40.822534 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.822414    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2rfj\" (UniqueName: \"kubernetes.io/projected/ecda580b-18a1-435d-bb7a-44f97d6afec4-kube-api-access-x2rfj\") pod \"limitador-limitador-54d6d988d5-ggt7p\" (UID: \"ecda580b-18a1-435d-bb7a-44f97d6afec4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:40.923828 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.923772    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ecda580b-18a1-435d-bb7a-44f97d6afec4-config-file\") pod \"limitador-limitador-54d6d988d5-ggt7p\" (UID: \"ecda580b-18a1-435d-bb7a-44f97d6afec4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:40.923828 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.923837    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2rfj\" (UniqueName: \"kubernetes.io/projected/ecda580b-18a1-435d-bb7a-44f97d6afec4-kube-api-access-x2rfj\") pod \"limitador-limitador-54d6d988d5-ggt7p\" (UID: \"ecda580b-18a1-435d-bb7a-44f97d6afec4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:40.924533 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.924505    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ecda580b-18a1-435d-bb7a-44f97d6afec4-config-file\") pod \"limitador-limitador-54d6d988d5-ggt7p\" (UID: \"ecda580b-18a1-435d-bb7a-44f97d6afec4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:40.934870 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:40.934843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2rfj\" (UniqueName: \"kubernetes.io/projected/ecda580b-18a1-435d-bb7a-44f97d6afec4-kube-api-access-x2rfj\") pod \"limitador-limitador-54d6d988d5-ggt7p\" (UID: \"ecda580b-18a1-435d-bb7a-44f97d6afec4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:41.069352 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.069253    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:41.205451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.205423    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:19:41.207290 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:19:41.207260    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecda580b_18a1_435d_bb7a_44f97d6afec4.slice/crio-2e16eaec58b14f8b05ee8efc8fea59a7d9a3a7f6d65e7232639f715c3560e9d7 WatchSource:0}: Error finding container 2e16eaec58b14f8b05ee8efc8fea59a7d9a3a7f6d65e7232639f715c3560e9d7: Status 404 returned error can't find the container with id 2e16eaec58b14f8b05ee8efc8fea59a7d9a3a7f6d65e7232639f715c3560e9d7
Jul 29 15:19:41.307435 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.307398    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-h76bt"]
Jul 29 15:19:41.311909 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.311887    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h76bt"
Jul 29 15:19:41.314069 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.314042    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2rnqq\""
Jul 29 15:19:41.317340 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.317321    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-h76bt"]
Jul 29 15:19:41.325969 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.325943    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5kw\" (UniqueName: \"kubernetes.io/projected/4935f998-4fd7-4de5-bdb5-766e6476e718-kube-api-access-dd5kw\") pod \"authorino-58cd955675-h76bt\" (UID: \"4935f998-4fd7-4de5-bdb5-766e6476e718\") " pod="kuadrant-system/authorino-58cd955675-h76bt"
Jul 29 15:19:41.427172 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.427126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5kw\" (UniqueName: \"kubernetes.io/projected/4935f998-4fd7-4de5-bdb5-766e6476e718-kube-api-access-dd5kw\") pod \"authorino-58cd955675-h76bt\" (UID: \"4935f998-4fd7-4de5-bdb5-766e6476e718\") " pod="kuadrant-system/authorino-58cd955675-h76bt"
Jul 29 15:19:41.436618 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.436593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5kw\" (UniqueName: \"kubernetes.io/projected/4935f998-4fd7-4de5-bdb5-766e6476e718-kube-api-access-dd5kw\") pod \"authorino-58cd955675-h76bt\" (UID: \"4935f998-4fd7-4de5-bdb5-766e6476e718\") " pod="kuadrant-system/authorino-58cd955675-h76bt"
Jul 29 15:19:41.471813 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.471780    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-sv7rt"]
Jul 29 15:19:41.475131 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.475113    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sv7rt"
Jul 29 15:19:41.482844 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.482822    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sv7rt"]
Jul 29 15:19:41.527573 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.527529    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4twbp\" (UniqueName: \"kubernetes.io/projected/224039ad-ddce-4043-8406-57b4ed3ff008-kube-api-access-4twbp\") pod \"authorino-75d556bd77-sv7rt\" (UID: \"224039ad-ddce-4043-8406-57b4ed3ff008\") " pod="kuadrant-system/authorino-75d556bd77-sv7rt"
Jul 29 15:19:41.591374 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.591293    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p" event={"ID":"ecda580b-18a1-435d-bb7a-44f97d6afec4","Type":"ContainerStarted","Data":"2e16eaec58b14f8b05ee8efc8fea59a7d9a3a7f6d65e7232639f715c3560e9d7"}
Jul 29 15:19:41.623434 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.623405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h76bt"
Jul 29 15:19:41.628440 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.628417    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4twbp\" (UniqueName: \"kubernetes.io/projected/224039ad-ddce-4043-8406-57b4ed3ff008-kube-api-access-4twbp\") pod \"authorino-75d556bd77-sv7rt\" (UID: \"224039ad-ddce-4043-8406-57b4ed3ff008\") " pod="kuadrant-system/authorino-75d556bd77-sv7rt"
Jul 29 15:19:41.636717 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.636696    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4twbp\" (UniqueName: \"kubernetes.io/projected/224039ad-ddce-4043-8406-57b4ed3ff008-kube-api-access-4twbp\") pod \"authorino-75d556bd77-sv7rt\" (UID: \"224039ad-ddce-4043-8406-57b4ed3ff008\") " pod="kuadrant-system/authorino-75d556bd77-sv7rt"
Jul 29 15:19:41.786150 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.785884    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sv7rt"
Jul 29 15:19:41.805671 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.805610    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-h76bt"]
Jul 29 15:19:41.810946 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:19:41.810911    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4935f998_4fd7_4de5_bdb5_766e6476e718.slice/crio-b2f75d716ca6fb3a03fd4b5e2fce1ffd4e1e4c34ca7d29412d695b44ccff434e WatchSource:0}: Error finding container b2f75d716ca6fb3a03fd4b5e2fce1ffd4e1e4c34ca7d29412d695b44ccff434e: Status 404 returned error can't find the container with id b2f75d716ca6fb3a03fd4b5e2fce1ffd4e1e4c34ca7d29412d695b44ccff434e
Jul 29 15:19:41.980596 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:41.979640    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sv7rt"]
Jul 29 15:19:41.983970 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:19:41.982881    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod224039ad_ddce_4043_8406_57b4ed3ff008.slice/crio-cb925360b99374ea5489b538182574fbfb8debba45bb40feb9c92a02f9999c53 WatchSource:0}: Error finding container cb925360b99374ea5489b538182574fbfb8debba45bb40feb9c92a02f9999c53: Status 404 returned error can't find the container with id cb925360b99374ea5489b538182574fbfb8debba45bb40feb9c92a02f9999c53
Jul 29 15:19:42.598596 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:42.598523    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h76bt" event={"ID":"4935f998-4fd7-4de5-bdb5-766e6476e718","Type":"ContainerStarted","Data":"b2f75d716ca6fb3a03fd4b5e2fce1ffd4e1e4c34ca7d29412d695b44ccff434e"}
Jul 29 15:19:42.599992 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:42.599959    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sv7rt" event={"ID":"224039ad-ddce-4043-8406-57b4ed3ff008","Type":"ContainerStarted","Data":"cb925360b99374ea5489b538182574fbfb8debba45bb40feb9c92a02f9999c53"}
Jul 29 15:19:44.621613 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:44.621574    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p" event={"ID":"ecda580b-18a1-435d-bb7a-44f97d6afec4","Type":"ContainerStarted","Data":"eeb3689d46781d867be60161608264861c511e49d487ab4872d8ead55f05e659"}
Jul 29 15:19:44.622104 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:44.621775    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:19:44.642119 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:44.642070    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p" podStartSLOduration=1.6063842560000001 podStartE2EDuration="4.642054953s" totalImagesPullingTime="3.035670697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:44.639064375 +0000 UTC m=+522.933001189" watchObservedRunningTime="2026-07-29 15:19:44.642054953 +0000 UTC m=+522.935991754"
Jul 29 15:19:46.633955 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:46.633918    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sv7rt" event={"ID":"224039ad-ddce-4043-8406-57b4ed3ff008","Type":"ContainerStarted","Data":"51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd"}
Jul 29 15:19:46.635441 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:46.635399    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h76bt" event={"ID":"4935f998-4fd7-4de5-bdb5-766e6476e718","Type":"ContainerStarted","Data":"d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3"}
Jul 29 15:19:46.650351 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:46.650304    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-sv7rt" podStartSLOduration=1.5879516900000001 podStartE2EDuration="5.650290702s" totalImagesPullingTime="4.062339012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:46.648579724 +0000 UTC m=+524.942516526" watchObservedRunningTime="2026-07-29 15:19:46.650290702 +0000 UTC m=+524.944227504"
Jul 29 15:19:46.663409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:46.663360    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-h76bt" podStartSLOduration=1.417685126 podStartE2EDuration="5.663341029s" totalImagesPullingTime="4.245655903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:46.66233197 +0000 UTC m=+524.956268798" watchObservedRunningTime="2026-07-29 15:19:46.663341029 +0000 UTC m=+524.957277832"
Jul 29 15:19:46.686115 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:46.686086    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-h76bt"]
Jul 29 15:19:48.652304 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:48.652263    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-h76bt" podUID="4935f998-4fd7-4de5-bdb5-766e6476e718" containerName="authorino" containerID="cri-o://d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3" gracePeriod=30
Jul 29 15:19:48.907390 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:48.907323    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h76bt"
Jul 29 15:19:48.993912 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:48.993871    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd5kw\" (UniqueName: \"kubernetes.io/projected/4935f998-4fd7-4de5-bdb5-766e6476e718-kube-api-access-dd5kw\") pod \"4935f998-4fd7-4de5-bdb5-766e6476e718\" (UID: \"4935f998-4fd7-4de5-bdb5-766e6476e718\") "
Jul 29 15:19:48.995978 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:48.995948    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4935f998-4fd7-4de5-bdb5-766e6476e718-kube-api-access-dd5kw" (OuterVolumeSpecName: "kube-api-access-dd5kw") pod "4935f998-4fd7-4de5-bdb5-766e6476e718" (UID: "4935f998-4fd7-4de5-bdb5-766e6476e718"). InnerVolumeSpecName "kube-api-access-dd5kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:19:49.095280 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.095167    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd5kw\" (UniqueName: \"kubernetes.io/projected/4935f998-4fd7-4de5-bdb5-766e6476e718-kube-api-access-dd5kw\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:19:49.659077 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.659040    2560 generic.go:356] "Generic (PLEG): container finished" podID="4935f998-4fd7-4de5-bdb5-766e6476e718" containerID="d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3" exitCode=0
Jul 29 15:19:49.659545 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.659092    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h76bt"
Jul 29 15:19:49.659545 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.659130    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h76bt" event={"ID":"4935f998-4fd7-4de5-bdb5-766e6476e718","Type":"ContainerDied","Data":"d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3"}
Jul 29 15:19:49.659545 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.659193    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h76bt" event={"ID":"4935f998-4fd7-4de5-bdb5-766e6476e718","Type":"ContainerDied","Data":"b2f75d716ca6fb3a03fd4b5e2fce1ffd4e1e4c34ca7d29412d695b44ccff434e"}
Jul 29 15:19:49.659545 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.659215    2560 scope.go:117] "RemoveContainer" containerID="d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3"
Jul 29 15:19:49.668139 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.668118    2560 scope.go:117] "RemoveContainer" containerID="d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3"
Jul 29 15:19:49.668433 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:19:49.668413    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3\": container with ID starting with d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3 not found: ID does not exist" containerID="d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3"
Jul 29 15:19:49.668493 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.668443    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3"} err="failed to get container status \"d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3\": rpc error: code = NotFound desc = could not find container \"d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3\": container with ID starting with d7cf007f3a460c32c9014fc0cfe99a916b73f6550913f5048ed708d885d14ac3 not found: ID does not exist"
Jul 29 15:19:49.681031 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.681001    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-h76bt"]
Jul 29 15:19:49.687690 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:49.687663    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-h76bt"]
Jul 29 15:19:50.166077 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:50.166042    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4935f998-4fd7-4de5-bdb5-766e6476e718" path="/var/lib/kubelet/pods/4935f998-4fd7-4de5-bdb5-766e6476e718/volumes"
Jul 29 15:19:55.628598 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:19:55.628519    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"
Jul 29 15:20:14.259989 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.259946    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzwnt"]
Jul 29 15:20:14.260406 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.260352    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4935f998-4fd7-4de5-bdb5-766e6476e718" containerName="authorino"
Jul 29 15:20:14.260406 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.260365    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4935f998-4fd7-4de5-bdb5-766e6476e718" containerName="authorino"
Jul 29 15:20:14.260493 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.260441    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4935f998-4fd7-4de5-bdb5-766e6476e718" containerName="authorino"
Jul 29 15:20:14.264788 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.264742    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzwnt"
Jul 29 15:20:14.267567 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.267537    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzwnt"]
Jul 29 15:20:14.419728 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.419674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dxvr\" (UniqueName: \"kubernetes.io/projected/328e928e-90be-4350-b8c6-4fa0d660df8e-kube-api-access-5dxvr\") pod \"authorino-5d4b67c446-xzwnt\" (UID: \"328e928e-90be-4350-b8c6-4fa0d660df8e\") " pod="kuadrant-system/authorino-5d4b67c446-xzwnt"
Jul 29 15:20:14.520265 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.520156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dxvr\" (UniqueName: \"kubernetes.io/projected/328e928e-90be-4350-b8c6-4fa0d660df8e-kube-api-access-5dxvr\") pod \"authorino-5d4b67c446-xzwnt\" (UID: \"328e928e-90be-4350-b8c6-4fa0d660df8e\") " pod="kuadrant-system/authorino-5d4b67c446-xzwnt"
Jul 29 15:20:14.528945 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.528922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dxvr\" (UniqueName: \"kubernetes.io/projected/328e928e-90be-4350-b8c6-4fa0d660df8e-kube-api-access-5dxvr\") pod \"authorino-5d4b67c446-xzwnt\" (UID: \"328e928e-90be-4350-b8c6-4fa0d660df8e\") " pod="kuadrant-system/authorino-5d4b67c446-xzwnt"
Jul 29 15:20:14.532594 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.532568    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzwnt"]
Jul 29 15:20:14.532805 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.532793    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzwnt"
Jul 29 15:20:14.563027 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.562999    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d78b6b854-prklz"]
Jul 29 15:20:14.566667 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.566646    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d78b6b854-prklz"
Jul 29 15:20:14.575161 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.575126    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d78b6b854-prklz"]
Jul 29 15:20:14.629140 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.629109    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d78b6b854-prklz"]
Jul 29 15:20:14.629407 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:20:14.629385    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9jvch], unattached volumes=[], failed to process volumes=[kube-api-access-9jvch]: context canceled" pod="kuadrant-system/authorino-6d78b6b854-prklz" podUID="85f21ae6-87e3-471f-8863-aa8d98c7a53a"
Jul 29 15:20:14.656028 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.655974    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5845c48f86-kzp24"]
Jul 29 15:20:14.659378 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.659362    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:14.662138 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.662120    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 15:20:14.668669 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.668649    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5845c48f86-kzp24"]
Jul 29 15:20:14.678108 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.678040    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzwnt"]
Jul 29 15:20:14.680807 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:20:14.680783    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod328e928e_90be_4350_b8c6_4fa0d660df8e.slice/crio-14df13b5f0751d6a8be91f38b304d63674814d0b56547cec04ed65ffac5c27e0 WatchSource:0}: Error finding container 14df13b5f0751d6a8be91f38b304d63674814d0b56547cec04ed65ffac5c27e0: Status 404 returned error can't find the container with id 14df13b5f0751d6a8be91f38b304d63674814d0b56547cec04ed65ffac5c27e0
Jul 29 15:20:14.721698 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.721669    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs4qn\" (UniqueName: \"kubernetes.io/projected/4c5cf545-4170-4fad-a99e-f098ec4cc36e-kube-api-access-fs4qn\") pod \"authorino-5845c48f86-kzp24\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") " pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:14.721858 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.721728    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c5cf545-4170-4fad-a99e-f098ec4cc36e-tls-cert\") pod \"authorino-5845c48f86-kzp24\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") " pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:14.721858 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.721777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jvch\" (UniqueName: \"kubernetes.io/projected/85f21ae6-87e3-471f-8863-aa8d98c7a53a-kube-api-access-9jvch\") pod \"authorino-6d78b6b854-prklz\" (UID: \"85f21ae6-87e3-471f-8863-aa8d98c7a53a\") " pod="kuadrant-system/authorino-6d78b6b854-prklz"
Jul 29 15:20:14.789070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.788970    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xzwnt" event={"ID":"328e928e-90be-4350-b8c6-4fa0d660df8e","Type":"ContainerStarted","Data":"14df13b5f0751d6a8be91f38b304d63674814d0b56547cec04ed65ffac5c27e0"}
Jul 29 15:20:14.789070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.788996    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d78b6b854-prklz"
Jul 29 15:20:14.796178 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.796156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d78b6b854-prklz"
Jul 29 15:20:14.822655 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.822625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs4qn\" (UniqueName: \"kubernetes.io/projected/4c5cf545-4170-4fad-a99e-f098ec4cc36e-kube-api-access-fs4qn\") pod \"authorino-5845c48f86-kzp24\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") " pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:14.822841 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.822781    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c5cf545-4170-4fad-a99e-f098ec4cc36e-tls-cert\") pod \"authorino-5845c48f86-kzp24\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") " pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:14.822841 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.822800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jvch\" (UniqueName: \"kubernetes.io/projected/85f21ae6-87e3-471f-8863-aa8d98c7a53a-kube-api-access-9jvch\") pod \"authorino-6d78b6b854-prklz\" (UID: \"85f21ae6-87e3-471f-8863-aa8d98c7a53a\") " pod="kuadrant-system/authorino-6d78b6b854-prklz"
Jul 29 15:20:14.825350 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.825326    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c5cf545-4170-4fad-a99e-f098ec4cc36e-tls-cert\") pod \"authorino-5845c48f86-kzp24\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") " pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:14.832058 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.832036    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs4qn\" (UniqueName: \"kubernetes.io/projected/4c5cf545-4170-4fad-a99e-f098ec4cc36e-kube-api-access-fs4qn\") pod \"authorino-5845c48f86-kzp24\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") " pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:14.832282 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.832263    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jvch\" (UniqueName: \"kubernetes.io/projected/85f21ae6-87e3-471f-8863-aa8d98c7a53a-kube-api-access-9jvch\") pod \"authorino-6d78b6b854-prklz\" (UID: \"85f21ae6-87e3-471f-8863-aa8d98c7a53a\") " pod="kuadrant-system/authorino-6d78b6b854-prklz"
Jul 29 15:20:14.923664 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.923614    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jvch\" (UniqueName: \"kubernetes.io/projected/85f21ae6-87e3-471f-8863-aa8d98c7a53a-kube-api-access-9jvch\") pod \"85f21ae6-87e3-471f-8863-aa8d98c7a53a\" (UID: \"85f21ae6-87e3-471f-8863-aa8d98c7a53a\") "
Jul 29 15:20:14.925677 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.925652    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f21ae6-87e3-471f-8863-aa8d98c7a53a-kube-api-access-9jvch" (OuterVolumeSpecName: "kube-api-access-9jvch") pod "85f21ae6-87e3-471f-8863-aa8d98c7a53a" (UID: "85f21ae6-87e3-471f-8863-aa8d98c7a53a"). InnerVolumeSpecName "kube-api-access-9jvch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:20:14.969221 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:14.969199    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:20:15.024622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.024588    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jvch\" (UniqueName: \"kubernetes.io/projected/85f21ae6-87e3-471f-8863-aa8d98c7a53a-kube-api-access-9jvch\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:20:15.103712 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.103687    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5845c48f86-kzp24"]
Jul 29 15:20:15.106985 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:20:15.106954    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c5cf545_4170_4fad_a99e_f098ec4cc36e.slice/crio-004655ec8ff81bd0b89c98dfc88af0f0ed134232446764840e79b4732d6676be WatchSource:0}: Error finding container 004655ec8ff81bd0b89c98dfc88af0f0ed134232446764840e79b4732d6676be: Status 404 returned error can't find the container with id 004655ec8ff81bd0b89c98dfc88af0f0ed134232446764840e79b4732d6676be
Jul 29 15:20:15.795626 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.795531    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xzwnt" event={"ID":"328e928e-90be-4350-b8c6-4fa0d660df8e","Type":"ContainerStarted","Data":"2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164"}
Jul 29 15:20:15.795626 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.795577    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-xzwnt" podUID="328e928e-90be-4350-b8c6-4fa0d660df8e" containerName="authorino" containerID="cri-o://2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164" gracePeriod=30
Jul 29 15:20:15.797289 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.797248    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5845c48f86-kzp24" event={"ID":"4c5cf545-4170-4fad-a99e-f098ec4cc36e","Type":"ContainerStarted","Data":"2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df"}
Jul 29 15:20:15.797428 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.797299    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d78b6b854-prklz"
Jul 29 15:20:15.797428 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.797301    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5845c48f86-kzp24" event={"ID":"4c5cf545-4170-4fad-a99e-f098ec4cc36e","Type":"ContainerStarted","Data":"004655ec8ff81bd0b89c98dfc88af0f0ed134232446764840e79b4732d6676be"}
Jul 29 15:20:15.810591 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.810533    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-xzwnt" podStartSLOduration=1.4393051510000001 podStartE2EDuration="1.810517225s" totalImagesPullingTime="371.212074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:15.810125815 +0000 UTC m=+554.104062619" watchObservedRunningTime="2026-07-29 15:20:15.810517225 +0000 UTC m=+554.104454022"
Jul 29 15:20:15.827473 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.827426    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5845c48f86-kzp24" podStartSLOduration=1.4552490599999999 podStartE2EDuration="1.827410685s" totalImagesPullingTime="372.161625ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:15.82524229 +0000 UTC m=+554.119179094" watchObservedRunningTime="2026-07-29 15:20:15.827410685 +0000 UTC m=+554.121347486"
Jul 29 15:20:15.854033 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.853994    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sv7rt"]
Jul 29 15:20:15.854227 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.854177    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-sv7rt" podUID="224039ad-ddce-4043-8406-57b4ed3ff008" containerName="authorino" containerID="cri-o://51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd" gracePeriod=30
Jul 29 15:20:15.893527 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.893500    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d78b6b854-prklz"]
Jul 29 15:20:15.898407 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:15.898381    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d78b6b854-prklz"]
Jul 29 15:20:16.108735 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.108711    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzwnt"
Jul 29 15:20:16.130911 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.130864    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sv7rt"
Jul 29 15:20:16.167846 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.167811    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f21ae6-87e3-471f-8863-aa8d98c7a53a" path="/var/lib/kubelet/pods/85f21ae6-87e3-471f-8863-aa8d98c7a53a/volumes"
Jul 29 15:20:16.234745 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.234716    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4twbp\" (UniqueName: \"kubernetes.io/projected/224039ad-ddce-4043-8406-57b4ed3ff008-kube-api-access-4twbp\") pod \"224039ad-ddce-4043-8406-57b4ed3ff008\" (UID: \"224039ad-ddce-4043-8406-57b4ed3ff008\") "
Jul 29 15:20:16.234904 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.234790    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dxvr\" (UniqueName: \"kubernetes.io/projected/328e928e-90be-4350-b8c6-4fa0d660df8e-kube-api-access-5dxvr\") pod \"328e928e-90be-4350-b8c6-4fa0d660df8e\" (UID: \"328e928e-90be-4350-b8c6-4fa0d660df8e\") "
Jul 29 15:20:16.236885 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.236859    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/224039ad-ddce-4043-8406-57b4ed3ff008-kube-api-access-4twbp" (OuterVolumeSpecName: "kube-api-access-4twbp") pod "224039ad-ddce-4043-8406-57b4ed3ff008" (UID: "224039ad-ddce-4043-8406-57b4ed3ff008"). InnerVolumeSpecName "kube-api-access-4twbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:20:16.236885 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.236873    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328e928e-90be-4350-b8c6-4fa0d660df8e-kube-api-access-5dxvr" (OuterVolumeSpecName: "kube-api-access-5dxvr") pod "328e928e-90be-4350-b8c6-4fa0d660df8e" (UID: "328e928e-90be-4350-b8c6-4fa0d660df8e"). InnerVolumeSpecName "kube-api-access-5dxvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:20:16.336432 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.336353    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4twbp\" (UniqueName: \"kubernetes.io/projected/224039ad-ddce-4043-8406-57b4ed3ff008-kube-api-access-4twbp\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:20:16.336432 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.336381    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dxvr\" (UniqueName: \"kubernetes.io/projected/328e928e-90be-4350-b8c6-4fa0d660df8e-kube-api-access-5dxvr\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:20:16.803835 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.803800    2560 generic.go:356] "Generic (PLEG): container finished" podID="224039ad-ddce-4043-8406-57b4ed3ff008" containerID="51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd" exitCode=0
Jul 29 15:20:16.804338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.803839    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sv7rt"
Jul 29 15:20:16.804338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.803891    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sv7rt" event={"ID":"224039ad-ddce-4043-8406-57b4ed3ff008","Type":"ContainerDied","Data":"51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd"}
Jul 29 15:20:16.804338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.803935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sv7rt" event={"ID":"224039ad-ddce-4043-8406-57b4ed3ff008","Type":"ContainerDied","Data":"cb925360b99374ea5489b538182574fbfb8debba45bb40feb9c92a02f9999c53"}
Jul 29 15:20:16.804338 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.803953    2560 scope.go:117] "RemoveContainer" containerID="51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd"
Jul 29 15:20:16.805472 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.805291    2560 generic.go:356] "Generic (PLEG): container finished" podID="328e928e-90be-4350-b8c6-4fa0d660df8e" containerID="2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164" exitCode=0
Jul 29 15:20:16.805472 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.805329    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzwnt"
Jul 29 15:20:16.805472 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.805327    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xzwnt" event={"ID":"328e928e-90be-4350-b8c6-4fa0d660df8e","Type":"ContainerDied","Data":"2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164"}
Jul 29 15:20:16.805472 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.805378    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xzwnt" event={"ID":"328e928e-90be-4350-b8c6-4fa0d660df8e","Type":"ContainerDied","Data":"14df13b5f0751d6a8be91f38b304d63674814d0b56547cec04ed65ffac5c27e0"}
Jul 29 15:20:16.813730 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.813712    2560 scope.go:117] "RemoveContainer" containerID="51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd"
Jul 29 15:20:16.814019 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:20:16.814000    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd\": container with ID starting with 51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd not found: ID does not exist" containerID="51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd"
Jul 29 15:20:16.814090 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.814026    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd"} err="failed to get container status \"51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd\": rpc error: code = NotFound desc = could not find container \"51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd\": container with ID starting with 51b3b94d24115bee1f7b091c6ea66e870fad77240aebfd30f2c15c533c5fa2bd not found: ID does not exist"
Jul 29 15:20:16.814090 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.814043    2560 scope.go:117] "RemoveContainer" containerID="2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164"
Jul 29 15:20:16.822057 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.822042    2560 scope.go:117] "RemoveContainer" containerID="2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164"
Jul 29 15:20:16.822325 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:20:16.822305    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164\": container with ID starting with 2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164 not found: ID does not exist" containerID="2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164"
Jul 29 15:20:16.822381 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.822331    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164"} err="failed to get container status \"2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164\": rpc error: code = NotFound desc = could not find container \"2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164\": container with ID starting with 2c278c2bce2115e09cad79df258c885573b69c98ee6711ba61f896c33d9a5164 not found: ID does not exist"
Jul 29 15:20:16.829062 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.829039    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzwnt"]
Jul 29 15:20:16.833652 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.833632    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzwnt"]
Jul 29 15:20:16.846270 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.846241    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sv7rt"]
Jul 29 15:20:16.853294 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:16.853270    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sv7rt"]
Jul 29 15:20:18.165734 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:18.165699    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="224039ad-ddce-4043-8406-57b4ed3ff008" path="/var/lib/kubelet/pods/224039ad-ddce-4043-8406-57b4ed3ff008/volumes"
Jul 29 15:20:18.166176 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:18.166022    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="328e928e-90be-4350-b8c6-4fa0d660df8e" path="/var/lib/kubelet/pods/328e928e-90be-4350-b8c6-4fa0d660df8e/volumes"
Jul 29 15:20:57.637059 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:20:57.637019    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:21:02.074943 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:02.074905    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:21:02.076718 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:02.076693    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:21:02.083300 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:02.083278    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:21:02.085041 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:02.085024    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:21:26.489957 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:26.489868    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:21:36.790166 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:36.790117    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:21:45.479966 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.479933    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"]
Jul 29 15:21:45.480390 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.480370    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328e928e-90be-4350-b8c6-4fa0d660df8e" containerName="authorino"
Jul 29 15:21:45.480438 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.480393    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="328e928e-90be-4350-b8c6-4fa0d660df8e" containerName="authorino"
Jul 29 15:21:45.480438 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.480419    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="224039ad-ddce-4043-8406-57b4ed3ff008" containerName="authorino"
Jul 29 15:21:45.480438 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.480427    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="224039ad-ddce-4043-8406-57b4ed3ff008" containerName="authorino"
Jul 29 15:21:45.480539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.480500    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="328e928e-90be-4350-b8c6-4fa0d660df8e" containerName="authorino"
Jul 29 15:21:45.480539 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.480508    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="224039ad-ddce-4043-8406-57b4ed3ff008" containerName="authorino"
Jul 29 15:21:45.483683 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.483664    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.486401 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.486380    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lc787\""
Jul 29 15:21:45.487139 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.487122    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 15:21:45.487300 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.487203    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 15:21:45.487300 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.487225    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 15:21:45.494861 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.494838    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"]
Jul 29 15:21:45.593541 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.593504    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.593733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.593583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bb1207-576a-448c-981b-3f4326059884-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.593733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.593606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.593733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.593626    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.593733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.593644    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.593733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.593680    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jfrf\" (UniqueName: \"kubernetes.io/projected/45bb1207-576a-448c-981b-3f4326059884-kube-api-access-9jfrf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694485 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694446    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694508    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bb1207-576a-448c-981b-3f4326059884-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694526    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694544    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694673 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jfrf\" (UniqueName: \"kubernetes.io/projected/45bb1207-576a-448c-981b-3f4326059884-kube-api-access-9jfrf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.694974 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694914    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.695024 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.694984    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.696830 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.696806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45bb1207-576a-448c-981b-3f4326059884-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.697163 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.697148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45bb1207-576a-448c-981b-3f4326059884-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.703436 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.703418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jfrf\" (UniqueName: \"kubernetes.io/projected/45bb1207-576a-448c-981b-3f4326059884-kube-api-access-9jfrf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwkhx\" (UID: \"45bb1207-576a-448c-981b-3f4326059884\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.794363 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.794273    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:45.932575 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.932544    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"]
Jul 29 15:21:45.934827 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:21:45.934800    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45bb1207_576a_448c_981b_3f4326059884.slice/crio-239debdd80a6a0e9ba69d1eeaae6dccd215753d6cd8bdfa78bb6e5cdd873820c WatchSource:0}: Error finding container 239debdd80a6a0e9ba69d1eeaae6dccd215753d6cd8bdfa78bb6e5cdd873820c: Status 404 returned error can't find the container with id 239debdd80a6a0e9ba69d1eeaae6dccd215753d6cd8bdfa78bb6e5cdd873820c
Jul 29 15:21:45.936299 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:45.936276    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:21:46.263819 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:46.263781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx" event={"ID":"45bb1207-576a-448c-981b-3f4326059884","Type":"ContainerStarted","Data":"239debdd80a6a0e9ba69d1eeaae6dccd215753d6cd8bdfa78bb6e5cdd873820c"}
Jul 29 15:21:46.782711 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:46.782674    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:21:48.274622 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:48.274581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx" event={"ID":"45bb1207-576a-448c-981b-3f4326059884","Type":"ContainerStarted","Data":"6fdc92706b197e990f1b0bd5b9ab55806fc01d7be72f3302871994a7a07c4c45"}
Jul 29 15:21:48.297548 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:48.297492    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx" podStartSLOduration=1.970252777 podStartE2EDuration="3.297473756s" totalImagesPullingTime="1.327220979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:21:48.296049985 +0000 UTC m=+646.589986802" watchObservedRunningTime="2026-07-29 15:21:48.297473756 +0000 UTC m=+646.591410558"
Jul 29 15:21:55.794747 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:55.794694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:55.795166 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:55.794777    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:55.807329 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:55.807309    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:21:56.326458 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:21:56.326425    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwkhx"
Jul 29 15:22:04.190177 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:04.190134    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:22:15.692517 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:15.692480    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:22:23.477610 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.477571    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"]
Jul 29 15:22:23.485090 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.485063    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.487704 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.487682    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 15:22:23.495415 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.495388    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"]
Jul 29 15:22:23.533542 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.533504    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.533738 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.533564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9339091-fba0-4dd6-8d99-dd054b9c5391-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.533738 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.533617    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.533738 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.533648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg68l\" (UniqueName: \"kubernetes.io/projected/e9339091-fba0-4dd6-8d99-dd054b9c5391-kube-api-access-hg68l\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.533738 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.533718    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.533881 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.533772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635054 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg68l\" (UniqueName: \"kubernetes.io/projected/e9339091-fba0-4dd6-8d99-dd054b9c5391-kube-api-access-hg68l\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635054 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635330 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635330 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635129    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635330 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9339091-fba0-4dd6-8d99-dd054b9c5391-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635492 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635467    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635557 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635533    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635624 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635587    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.635688 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.635620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.637689 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.637664    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9339091-fba0-4dd6-8d99-dd054b9c5391-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.637802 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.637779    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9339091-fba0-4dd6-8d99-dd054b9c5391-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.644339 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.644314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg68l\" (UniqueName: \"kubernetes.io/projected/e9339091-fba0-4dd6-8d99-dd054b9c5391-kube-api-access-hg68l\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt\" (UID: \"e9339091-fba0-4dd6-8d99-dd054b9c5391\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.795591 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.795505    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:23.931598 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:23.931573    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"]
Jul 29 15:22:23.933771 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:22:23.933748    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9339091_fba0_4dd6_8d99_dd054b9c5391.slice/crio-307c41e8915df8912a732c7ea7d49ffc2ff1f03f892aae59e07d85db361a68c4 WatchSource:0}: Error finding container 307c41e8915df8912a732c7ea7d49ffc2ff1f03f892aae59e07d85db361a68c4: Status 404 returned error can't find the container with id 307c41e8915df8912a732c7ea7d49ffc2ff1f03f892aae59e07d85db361a68c4
Jul 29 15:22:24.462579 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:24.462540    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt" event={"ID":"e9339091-fba0-4dd6-8d99-dd054b9c5391","Type":"ContainerStarted","Data":"307c41e8915df8912a732c7ea7d49ffc2ff1f03f892aae59e07d85db361a68c4"}
Jul 29 15:22:25.470529 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:25.470494    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt" event={"ID":"e9339091-fba0-4dd6-8d99-dd054b9c5391","Type":"ContainerStarted","Data":"7475413ae829e886e3549d06d1e93e0e0229d6d6495984d1b4ab1c4a58e85f5f"}
Jul 29 15:22:25.492566 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:25.492518    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt" podStartSLOduration=1.638906873 podStartE2EDuration="2.492502264s" totalImagesPullingTime="853.595391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:22:25.48926433 +0000 UTC m=+683.783201133" watchObservedRunningTime="2026-07-29 15:22:25.492502264 +0000 UTC m=+683.786439065"
Jul 29 15:22:27.392286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:27.392246    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:22:33.796642 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:33.796601    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:33.797065 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:33.796657    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:33.809880 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:33.809849    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:22:34.530289 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:22:34.530261    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt"
Jul 29 15:24:23.176197 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.176154    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7cb6d9696c-276l5"]
Jul 29 15:24:23.179586 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.179567    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.195645 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.195617    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cb6d9696c-276l5"]
Jul 29 15:24:23.304308 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.304266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs7vc\" (UniqueName: \"kubernetes.io/projected/a66ef0de-8bb6-49d2-8f8f-defa225bc09c-kube-api-access-vs7vc\") pod \"authorino-7cb6d9696c-276l5\" (UID: \"a66ef0de-8bb6-49d2-8f8f-defa225bc09c\") " pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.304491 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.304379    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a66ef0de-8bb6-49d2-8f8f-defa225bc09c-tls-cert\") pod \"authorino-7cb6d9696c-276l5\" (UID: \"a66ef0de-8bb6-49d2-8f8f-defa225bc09c\") " pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.405680 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.405639    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs7vc\" (UniqueName: \"kubernetes.io/projected/a66ef0de-8bb6-49d2-8f8f-defa225bc09c-kube-api-access-vs7vc\") pod \"authorino-7cb6d9696c-276l5\" (UID: \"a66ef0de-8bb6-49d2-8f8f-defa225bc09c\") " pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.405859 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.405698    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a66ef0de-8bb6-49d2-8f8f-defa225bc09c-tls-cert\") pod \"authorino-7cb6d9696c-276l5\" (UID: \"a66ef0de-8bb6-49d2-8f8f-defa225bc09c\") " pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.408258 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.408238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a66ef0de-8bb6-49d2-8f8f-defa225bc09c-tls-cert\") pod \"authorino-7cb6d9696c-276l5\" (UID: \"a66ef0de-8bb6-49d2-8f8f-defa225bc09c\") " pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.416047 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.416019    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs7vc\" (UniqueName: \"kubernetes.io/projected/a66ef0de-8bb6-49d2-8f8f-defa225bc09c-kube-api-access-vs7vc\") pod \"authorino-7cb6d9696c-276l5\" (UID: \"a66ef0de-8bb6-49d2-8f8f-defa225bc09c\") " pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.488247 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.488149    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cb6d9696c-276l5"
Jul 29 15:24:23.638535 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:23.638508    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cb6d9696c-276l5"]
Jul 29 15:24:23.641533 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:24:23.641496    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda66ef0de_8bb6_49d2_8f8f_defa225bc09c.slice/crio-1ce677a82a7082c172d6771e75b0d21f87149f8faaff742d818fff1a1c25a562 WatchSource:0}: Error finding container 1ce677a82a7082c172d6771e75b0d21f87149f8faaff742d818fff1a1c25a562: Status 404 returned error can't find the container with id 1ce677a82a7082c172d6771e75b0d21f87149f8faaff742d818fff1a1c25a562
Jul 29 15:24:24.109490 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:24.109451    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cb6d9696c-276l5" event={"ID":"a66ef0de-8bb6-49d2-8f8f-defa225bc09c","Type":"ContainerStarted","Data":"1ce677a82a7082c172d6771e75b0d21f87149f8faaff742d818fff1a1c25a562"}
Jul 29 15:24:25.120466 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.120420    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cb6d9696c-276l5" event={"ID":"a66ef0de-8bb6-49d2-8f8f-defa225bc09c","Type":"ContainerStarted","Data":"bf3911092f266595f503c0ac29e309d3dd1d1b82b6054bfe33dce606795a0376"}
Jul 29 15:24:25.145342 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.145285    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7cb6d9696c-276l5" podStartSLOduration=1.609792879 podStartE2EDuration="2.145265063s" totalImagesPullingTime="535.472184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:24:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:24:25.142759963 +0000 UTC m=+803.436696757" watchObservedRunningTime="2026-07-29 15:24:25.145265063 +0000 UTC m=+803.439201866"
Jul 29 15:24:25.189896 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.189859    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5845c48f86-kzp24"]
Jul 29 15:24:25.190325 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.190292    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5845c48f86-kzp24" podUID="4c5cf545-4170-4fad-a99e-f098ec4cc36e" containerName="authorino" containerID="cri-o://2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df" gracePeriod=30
Jul 29 15:24:25.451909 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.451881    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:24:25.525988 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.525951    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs4qn\" (UniqueName: \"kubernetes.io/projected/4c5cf545-4170-4fad-a99e-f098ec4cc36e-kube-api-access-fs4qn\") pod \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") "
Jul 29 15:24:25.526156 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.526015    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c5cf545-4170-4fad-a99e-f098ec4cc36e-tls-cert\") pod \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\" (UID: \"4c5cf545-4170-4fad-a99e-f098ec4cc36e\") "
Jul 29 15:24:25.528110 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.528078    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c5cf545-4170-4fad-a99e-f098ec4cc36e-kube-api-access-fs4qn" (OuterVolumeSpecName: "kube-api-access-fs4qn") pod "4c5cf545-4170-4fad-a99e-f098ec4cc36e" (UID: "4c5cf545-4170-4fad-a99e-f098ec4cc36e"). InnerVolumeSpecName "kube-api-access-fs4qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:24:25.537987 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.537960    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c5cf545-4170-4fad-a99e-f098ec4cc36e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4c5cf545-4170-4fad-a99e-f098ec4cc36e" (UID: "4c5cf545-4170-4fad-a99e-f098ec4cc36e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:24:25.627640 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.627603    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c5cf545-4170-4fad-a99e-f098ec4cc36e-tls-cert\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:24:25.627640 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:25.627637    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs4qn\" (UniqueName: \"kubernetes.io/projected/4c5cf545-4170-4fad-a99e-f098ec4cc36e-kube-api-access-fs4qn\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:24:26.127733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.127698    2560 generic.go:356] "Generic (PLEG): container finished" podID="4c5cf545-4170-4fad-a99e-f098ec4cc36e" containerID="2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df" exitCode=0
Jul 29 15:24:26.128267 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.127743    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5845c48f86-kzp24"
Jul 29 15:24:26.128267 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.127789    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5845c48f86-kzp24" event={"ID":"4c5cf545-4170-4fad-a99e-f098ec4cc36e","Type":"ContainerDied","Data":"2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df"}
Jul 29 15:24:26.128267 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.127841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5845c48f86-kzp24" event={"ID":"4c5cf545-4170-4fad-a99e-f098ec4cc36e","Type":"ContainerDied","Data":"004655ec8ff81bd0b89c98dfc88af0f0ed134232446764840e79b4732d6676be"}
Jul 29 15:24:26.128267 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.127865    2560 scope.go:117] "RemoveContainer" containerID="2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df"
Jul 29 15:24:26.137384 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.137363    2560 scope.go:117] "RemoveContainer" containerID="2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df"
Jul 29 15:24:26.137661 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:24:26.137643    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df\": container with ID starting with 2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df not found: ID does not exist" containerID="2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df"
Jul 29 15:24:26.137728 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.137677    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df"} err="failed to get container status \"2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df\": rpc error: code = NotFound desc = could not find container \"2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df\": container with ID starting with 2772b1d8a30dc061afa000995543fee231c81d1af5d9eea115d0a00ac1b079df not found: ID does not exist"
Jul 29 15:24:26.152406 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.152375    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5845c48f86-kzp24"]
Jul 29 15:24:26.160683 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.160652    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5845c48f86-kzp24"]
Jul 29 15:24:26.167390 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:26.167364    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c5cf545-4170-4fad-a99e-f098ec4cc36e" path="/var/lib/kubelet/pods/4c5cf545-4170-4fad-a99e-f098ec4cc36e/volumes"
Jul 29 15:24:54.784453 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:24:54.784411    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:25:04.285321 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:25:04.285281    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:25:15.699567 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:25:15.699528    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:26:02.113637 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:26:02.113548    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:26:02.116811 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:26:02.116783    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:26:02.122368 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:26:02.122345    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:26:02.125409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:26:02.125088    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:26:59.892178 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:26:59.892138    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:27:08.184856 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:27:08.184820    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:28:21.980803 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:28:21.980769    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:28:31.187460 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:28:31.187421    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:28:46.083800 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:28:46.083759    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:28:55.582442 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:28:55.582401    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:29:06.993211 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:29:06.993156    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:29:17.280138 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:29:17.280101    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:29:27.688715 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:29:27.688675    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:29:37.382596 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:29:37.382557    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:30:00.393612 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:30:00.393572    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:30:11.779042 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:30:11.779009    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:30:35.886821 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:30:35.886736    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:30:49.385493 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:30:49.385456    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:31:02.148406 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:02.148372    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:31:02.153830 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:02.153804    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:31:02.156988 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:02.156961    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:31:02.162332 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:02.162312    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:31:03.383143 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:03.383102    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:31:24.318992 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:24.318953    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:31:39.495278 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:39.495240    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:31:48.190068 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:48.190028    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:31:59.788630 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:31:59.788543    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:32:08.585007 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:32:08.584973    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:32:16.985861 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:32:16.985824    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:32:39.694699 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:32:39.694663    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:32:51.092287 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:32:51.092250    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:32:58.986993 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:32:58.986956    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:35:12.287464 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:35:12.287425    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:35:22.091481 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:35:22.091445    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:35:44.389507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:35:44.389463    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:35:53.179790 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:35:53.179747    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:36:01.378562 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:01.378524    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:36:02.182994 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:02.182965    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:36:02.191430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:02.191404    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:36:02.197946 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:02.197918    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:36:02.206500 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:02.206476    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:36:11.076674 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:11.076634    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:36:19.983072 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:19.983032    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:36:30.120363 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:30.120267    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:36:40.883615 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:40.883577    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:36:50.884538 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:36:50.884503    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:37:01.025329 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:37:01.025293    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:37:11.274635 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:37:11.274602    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:37:36.679263 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:37:36.679223    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:37:44.082900 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:37:44.082862    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:38:20.797489 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.797452    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"]
Jul 29 15:38:20.797960 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.797814    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c5cf545-4170-4fad-a99e-f098ec4cc36e" containerName="authorino"
Jul 29 15:38:20.797960 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.797825    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c5cf545-4170-4fad-a99e-f098ec4cc36e" containerName="authorino"
Jul 29 15:38:20.797960 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.797884    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c5cf545-4170-4fad-a99e-f098ec4cc36e" containerName="authorino"
Jul 29 15:38:20.800966 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.800947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:20.803582 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.803561    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-907b5f75-kserve-self-signed-certs\""
Jul 29 15:38:20.822292 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.822269    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"]
Jul 29 15:38:20.907465 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.907419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-tmp-dir\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:20.907465 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.907460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-home\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:20.907687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.907485    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-model-cache\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:20.907687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.907551    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c561e98-d378-4555-9108-85de6c42de20-tls-certs\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:20.907687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.907626    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdds5\" (UniqueName: \"kubernetes.io/projected/7c561e98-d378-4555-9108-85de6c42de20-kube-api-access-fdds5\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:20.907687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:20.907659    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-dshm\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.008937 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.008899    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-tmp-dir\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.008937 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.008939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-home\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.009238 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.008960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-model-cache\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.009238 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.008992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c561e98-d378-4555-9108-85de6c42de20-tls-certs\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.009238 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.009027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdds5\" (UniqueName: \"kubernetes.io/projected/7c561e98-d378-4555-9108-85de6c42de20-kube-api-access-fdds5\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.009238 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.009047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-dshm\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.009465 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.009349    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-model-cache\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.009465 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.009434    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-tmp-dir\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.009544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.009463    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-home\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.011371 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.011351    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-dshm\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.011640 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.011621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c561e98-d378-4555-9108-85de6c42de20-tls-certs\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.018609 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.018585    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdds5\" (UniqueName: \"kubernetes.io/projected/7c561e98-d378-4555-9108-85de6c42de20-kube-api-access-fdds5\") pod \"e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") " pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.109844 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.109808    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:21.246560 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.246531    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"]
Jul 29 15:38:21.248819 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:38:21.248785    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c561e98_d378_4555_9108_85de6c42de20.slice/crio-37816cef45b8511b4113bf03e49360fef3cf9a73cb4293ba7f2abedce9b1f5bd WatchSource:0}: Error finding container 37816cef45b8511b4113bf03e49360fef3cf9a73cb4293ba7f2abedce9b1f5bd: Status 404 returned error can't find the container with id 37816cef45b8511b4113bf03e49360fef3cf9a73cb4293ba7f2abedce9b1f5bd
Jul 29 15:38:21.539368 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.539276    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w" event={"ID":"7c561e98-d378-4555-9108-85de6c42de20","Type":"ContainerStarted","Data":"53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3"}
Jul 29 15:38:21.539368 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.539319    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w" event={"ID":"7c561e98-d378-4555-9108-85de6c42de20","Type":"ContainerStarted","Data":"37816cef45b8511b4113bf03e49360fef3cf9a73cb4293ba7f2abedce9b1f5bd"}
Jul 29 15:38:21.558360 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:21.558306    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w" podStartSLOduration=1.5582882420000002 podStartE2EDuration="1.558288242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:38:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:38:21.556603046 +0000 UTC m=+1639.850539849" watchObservedRunningTime="2026-07-29 15:38:21.558288242 +0000 UTC m=+1639.852225043"
Jul 29 15:38:23.785153 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:23.785114    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:38:31.110690 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:31.110650    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:31.111092 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:31.110742    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:31.123031 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:31.123003    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:31.603225 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:31.603178    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:38:39.093492 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.093460    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"]
Jul 29 15:38:39.096925 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.096907    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.099212 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.099171    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-907b5f75-kserve-self-signed-certs\""
Jul 29 15:38:39.107743 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.107720    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"]
Jul 29 15:38:39.172643 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.172609    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/418026b8-8551-496e-9496-31792181455d-tls-certs\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.172849 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.172653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-model-cache\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.172849 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.172721    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-home\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.172849 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.172765    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-dshm\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.172849 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.172801    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-tmp-dir\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.173032 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.172898    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9cbq\" (UniqueName: \"kubernetes.io/projected/418026b8-8551-496e-9496-31792181455d-kube-api-access-v9cbq\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.273967 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.273924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-tmp-dir\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274150 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274001    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cbq\" (UniqueName: \"kubernetes.io/projected/418026b8-8551-496e-9496-31792181455d-kube-api-access-v9cbq\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274150 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/418026b8-8551-496e-9496-31792181455d-tls-certs\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274150 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274076    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-model-cache\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274417 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-home\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274417 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274274    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-dshm\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274417 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-tmp-dir\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274562 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-model-cache\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.274562 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.274531    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-home\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.276513 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.276490    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-dshm\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.276747 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.276730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/418026b8-8551-496e-9496-31792181455d-tls-certs\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.283006 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.282986    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9cbq\" (UniqueName: \"kubernetes.io/projected/418026b8-8551-496e-9496-31792181455d-kube-api-access-v9cbq\") pod \"e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b\" (UID: \"418026b8-8551-496e-9496-31792181455d\") " pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.407600 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.407504    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:39.549583 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.549557    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"]
Jul 29 15:38:39.551806 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:38:39.551780    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod418026b8_8551_496e_9496_31792181455d.slice/crio-f39b4c583dc89cafc4f41019c464247d926e3cf405fa433fabe467366962060c WatchSource:0}: Error finding container f39b4c583dc89cafc4f41019c464247d926e3cf405fa433fabe467366962060c: Status 404 returned error can't find the container with id f39b4c583dc89cafc4f41019c464247d926e3cf405fa433fabe467366962060c
Jul 29 15:38:39.634818 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.634778    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b" event={"ID":"418026b8-8551-496e-9496-31792181455d","Type":"ContainerStarted","Data":"c75f93cdf9a604cae0b10ed30d61d5f0e463ca34abe9bb58c376e940ca6146bb"}
Jul 29 15:38:39.635019 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.634829    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b" event={"ID":"418026b8-8551-496e-9496-31792181455d","Type":"ContainerStarted","Data":"f39b4c583dc89cafc4f41019c464247d926e3cf405fa433fabe467366962060c"}
Jul 29 15:38:39.653873 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:39.653818    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b" podStartSLOduration=0.653801979 podStartE2EDuration="653.801979ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:38:39.651487718 +0000 UTC m=+1657.945424512" watchObservedRunningTime="2026-07-29 15:38:39.653801979 +0000 UTC m=+1657.947738781"
Jul 29 15:38:42.581073 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:42.581036    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:38:49.407605 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:49.407573    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:49.408143 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:49.407620    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:49.420555 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:49.420519    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:38:49.704238 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:38:49.704132    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:39:06.075366 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:06.075328    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"]
Jul 29 15:39:07.688060 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:07.688013    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"]
Jul 29 15:39:09.286039 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:09.285995    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:39:21.095154 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.095095    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w" podUID="7c561e98-d378-4555-9108-85de6c42de20" containerName="main" containerID="cri-o://53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3" gracePeriod=45
Jul 29 15:39:21.349169 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.349100    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:39:21.475954 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.475923    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-dshm\") pod \"7c561e98-d378-4555-9108-85de6c42de20\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") "
Jul 29 15:39:21.475954 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.475959    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-tmp-dir\") pod \"7c561e98-d378-4555-9108-85de6c42de20\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") "
Jul 29 15:39:21.476275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.475981    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c561e98-d378-4555-9108-85de6c42de20-tls-certs\") pod \"7c561e98-d378-4555-9108-85de6c42de20\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") "
Jul 29 15:39:21.476275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476087    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-model-cache\") pod \"7c561e98-d378-4555-9108-85de6c42de20\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") "
Jul 29 15:39:21.476275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476118    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdds5\" (UniqueName: \"kubernetes.io/projected/7c561e98-d378-4555-9108-85de6c42de20-kube-api-access-fdds5\") pod \"7c561e98-d378-4555-9108-85de6c42de20\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") "
Jul 29 15:39:21.476275 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476172    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-home\") pod \"7c561e98-d378-4555-9108-85de6c42de20\" (UID: \"7c561e98-d378-4555-9108-85de6c42de20\") "
Jul 29 15:39:21.476504 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476311    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c561e98-d378-4555-9108-85de6c42de20" (UID: "7c561e98-d378-4555-9108-85de6c42de20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:21.476504 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476408    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c561e98-d378-4555-9108-85de6c42de20" (UID: "7c561e98-d378-4555-9108-85de6c42de20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:21.476504 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-home" (OuterVolumeSpecName: "home") pod "7c561e98-d378-4555-9108-85de6c42de20" (UID: "7c561e98-d378-4555-9108-85de6c42de20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:21.476661 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476524    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:21.476661 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.476544    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:21.478457 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.478435    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-dshm" (OuterVolumeSpecName: "dshm") pod "7c561e98-d378-4555-9108-85de6c42de20" (UID: "7c561e98-d378-4555-9108-85de6c42de20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:21.478558 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.478484    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c561e98-d378-4555-9108-85de6c42de20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c561e98-d378-4555-9108-85de6c42de20" (UID: "7c561e98-d378-4555-9108-85de6c42de20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:39:21.478558 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.478486    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c561e98-d378-4555-9108-85de6c42de20-kube-api-access-fdds5" (OuterVolumeSpecName: "kube-api-access-fdds5") pod "7c561e98-d378-4555-9108-85de6c42de20" (UID: "7c561e98-d378-4555-9108-85de6c42de20"). InnerVolumeSpecName "kube-api-access-fdds5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:39:21.577714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.577675    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdds5\" (UniqueName: \"kubernetes.io/projected/7c561e98-d378-4555-9108-85de6c42de20-kube-api-access-fdds5\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:21.577714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.577706    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:21.577714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.577717    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c561e98-d378-4555-9108-85de6c42de20-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:21.577714 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.577725    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c561e98-d378-4555-9108-85de6c42de20-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:21.864593 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.864562    2560 generic.go:356] "Generic (PLEG): container finished" podID="7c561e98-d378-4555-9108-85de6c42de20" containerID="53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3" exitCode=0
Jul 29 15:39:21.864775 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.864601    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"
Jul 29 15:39:21.864775 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.864611    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w" event={"ID":"7c561e98-d378-4555-9108-85de6c42de20","Type":"ContainerDied","Data":"53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3"}
Jul 29 15:39:21.864775 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.864643    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w" event={"ID":"7c561e98-d378-4555-9108-85de6c42de20","Type":"ContainerDied","Data":"37816cef45b8511b4113bf03e49360fef3cf9a73cb4293ba7f2abedce9b1f5bd"}
Jul 29 15:39:21.864775 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.864661    2560 scope.go:117] "RemoveContainer" containerID="53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3"
Jul 29 15:39:21.874341 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.874325    2560 scope.go:117] "RemoveContainer" containerID="53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3"
Jul 29 15:39:21.874605 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:39:21.874579    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3\": container with ID starting with 53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3 not found: ID does not exist" containerID="53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3"
Jul 29 15:39:21.874672 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.874617    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3"} err="failed to get container status \"53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3\": rpc error: code = NotFound desc = could not find container \"53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3\": container with ID starting with 53d9454e13856bdc1e34298d322079d29736b1c25783fd3cd3cc3ea213b892c3 not found: ID does not exist"
Jul 29 15:39:21.899561 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.899530    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"]
Jul 29 15:39:21.904223 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:21.904198    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-907b5f75-kserve-b88ddf4f9-smz4w"]
Jul 29 15:39:22.166145 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:22.166062    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c561e98-d378-4555-9108-85de6c42de20" path="/var/lib/kubelet/pods/7c561e98-d378-4555-9108-85de6c42de20/volumes"
Jul 29 15:39:22.707017 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:22.706969    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b" podUID="418026b8-8551-496e-9496-31792181455d" containerName="main" containerID="cri-o://c75f93cdf9a604cae0b10ed30d61d5f0e463ca34abe9bb58c376e940ca6146bb" gracePeriod=45
Jul 29 15:39:22.874166 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:22.874131    2560 generic.go:356] "Generic (PLEG): container finished" podID="418026b8-8551-496e-9496-31792181455d" containerID="c75f93cdf9a604cae0b10ed30d61d5f0e463ca34abe9bb58c376e940ca6146bb" exitCode=0
Jul 29 15:39:22.874344 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:22.874215    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b" event={"ID":"418026b8-8551-496e-9496-31792181455d","Type":"ContainerDied","Data":"c75f93cdf9a604cae0b10ed30d61d5f0e463ca34abe9bb58c376e940ca6146bb"}
Jul 29 15:39:22.958771 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:22.958711    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:39:23.090296 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090260    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-dshm\") pod \"418026b8-8551-496e-9496-31792181455d\" (UID: \"418026b8-8551-496e-9496-31792181455d\") "
Jul 29 15:39:23.090494 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090319    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/418026b8-8551-496e-9496-31792181455d-tls-certs\") pod \"418026b8-8551-496e-9496-31792181455d\" (UID: \"418026b8-8551-496e-9496-31792181455d\") "
Jul 29 15:39:23.090494 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090339    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-home\") pod \"418026b8-8551-496e-9496-31792181455d\" (UID: \"418026b8-8551-496e-9496-31792181455d\") "
Jul 29 15:39:23.090494 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090357    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9cbq\" (UniqueName: \"kubernetes.io/projected/418026b8-8551-496e-9496-31792181455d-kube-api-access-v9cbq\") pod \"418026b8-8551-496e-9496-31792181455d\" (UID: \"418026b8-8551-496e-9496-31792181455d\") "
Jul 29 15:39:23.090494 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090422    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-tmp-dir\") pod \"418026b8-8551-496e-9496-31792181455d\" (UID: \"418026b8-8551-496e-9496-31792181455d\") "
Jul 29 15:39:23.090494 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090445    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-model-cache\") pod \"418026b8-8551-496e-9496-31792181455d\" (UID: \"418026b8-8551-496e-9496-31792181455d\") "
Jul 29 15:39:23.090788 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090681    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "418026b8-8551-496e-9496-31792181455d" (UID: "418026b8-8551-496e-9496-31792181455d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:23.090788 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090693    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-home" (OuterVolumeSpecName: "home") pod "418026b8-8551-496e-9496-31792181455d" (UID: "418026b8-8551-496e-9496-31792181455d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:23.090788 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.090718    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-model-cache" (OuterVolumeSpecName: "model-cache") pod "418026b8-8551-496e-9496-31792181455d" (UID: "418026b8-8551-496e-9496-31792181455d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:23.092544 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.092514    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-dshm" (OuterVolumeSpecName: "dshm") pod "418026b8-8551-496e-9496-31792181455d" (UID: "418026b8-8551-496e-9496-31792181455d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:39:23.092670 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.092598    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/418026b8-8551-496e-9496-31792181455d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "418026b8-8551-496e-9496-31792181455d" (UID: "418026b8-8551-496e-9496-31792181455d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:39:23.092670 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.092653    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/418026b8-8551-496e-9496-31792181455d-kube-api-access-v9cbq" (OuterVolumeSpecName: "kube-api-access-v9cbq") pod "418026b8-8551-496e-9496-31792181455d" (UID: "418026b8-8551-496e-9496-31792181455d"). InnerVolumeSpecName "kube-api-access-v9cbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:39:23.191687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.191649    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:23.191687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.191684    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:23.191687 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.191693    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:23.192162 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.191701    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/418026b8-8551-496e-9496-31792181455d-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:23.192162 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.191709    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/418026b8-8551-496e-9496-31792181455d-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:23.192162 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.191717    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9cbq\" (UniqueName: \"kubernetes.io/projected/418026b8-8551-496e-9496-31792181455d-kube-api-access-v9cbq\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:39:23.880430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.880337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b" event={"ID":"418026b8-8551-496e-9496-31792181455d","Type":"ContainerDied","Data":"f39b4c583dc89cafc4f41019c464247d926e3cf405fa433fabe467366962060c"}
Jul 29 15:39:23.880430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.880369    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"
Jul 29 15:39:23.880430 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.880394    2560 scope.go:117] "RemoveContainer" containerID="c75f93cdf9a604cae0b10ed30d61d5f0e463ca34abe9bb58c376e940ca6146bb"
Jul 29 15:39:23.906483 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.906427    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"]
Jul 29 15:39:23.908895 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:23.908869    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-907b5f75-kserve-644cc6b95f-j8d7b"]
Jul 29 15:39:24.166706 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:24.166621    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="418026b8-8551-496e-9496-31792181455d" path="/var/lib/kubelet/pods/418026b8-8551-496e-9496-31792181455d/volumes"
Jul 29 15:39:24.194320 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:24.194289    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:39:30.899621 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:30.899579    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:39:41.281630 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:41.281583    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:39:47.790397 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:39:47.790356    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:40:00.793930 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:40:00.793892    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:40:14.904111 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:40:14.904065    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:40:20.497102 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:40:20.497065    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:40:29.794769 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:40:29.794733    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:40:38.489031 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:40:38.488991    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:40:52.989661 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:40:52.989618    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:41:02.210256 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:02.210216    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:41:02.224973 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:02.224947    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:41:02.232751 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:02.232726    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:41:02.236005 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:02.235983    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:41:02.247229 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:02.247209    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:41:12.100422 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:12.100383    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:41:24.603615 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:24.603574    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:41:34.905103 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:34.905054    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:41:41.194023 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:41.193987    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:41:52.192874 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:41:52.192840    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:42:07.681423 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:42:07.681386    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:42:20.984563 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:42:20.984522    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:42:35.592663 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:42:35.592576    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:42:51.187087 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:42:51.187052    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:43:00.390469 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:43:00.390429    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:43:13.683064 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:43:13.683029    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:43:28.185110 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:43:28.185074    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:43:38.098104 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:43:38.098067    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:43:48.201192 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:43:48.201153    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:43:56.382935 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:43:56.382836    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:44:18.661389 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.661354    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"]
Jul 29 15:44:18.661880 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.661708    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="418026b8-8551-496e-9496-31792181455d" containerName="main"
Jul 29 15:44:18.661880 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.661719    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="418026b8-8551-496e-9496-31792181455d" containerName="main"
Jul 29 15:44:18.661880 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.661737    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c561e98-d378-4555-9108-85de6c42de20" containerName="main"
Jul 29 15:44:18.661880 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.661743    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c561e98-d378-4555-9108-85de6c42de20" containerName="main"
Jul 29 15:44:18.661880 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.661799    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c561e98-d378-4555-9108-85de6c42de20" containerName="main"
Jul 29 15:44:18.661880 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.661808    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="418026b8-8551-496e-9496-31792181455d" containerName="main"
Jul 29 15:44:18.664806 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.664786    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"
Jul 29 15:44:18.667528 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.667501    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 15:44:18.667666 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.667505    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 15:44:18.677653 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.677623    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"]
Jul 29 15:44:18.749095 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.749053    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d77471e-654b-491c-b01b-dcfb11ca097f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-93e16c15-62jbw\" (UID: \"2d77471e-654b-491c-b01b-dcfb11ca097f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"
Jul 29 15:44:18.850060 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.850020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d77471e-654b-491c-b01b-dcfb11ca097f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-93e16c15-62jbw\" (UID: \"2d77471e-654b-491c-b01b-dcfb11ca097f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"
Jul 29 15:44:18.850688 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.850663    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d77471e-654b-491c-b01b-dcfb11ca097f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-93e16c15-62jbw\" (UID: \"2d77471e-654b-491c-b01b-dcfb11ca097f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"
Jul 29 15:44:18.977656 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:18.977554    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"
Jul 29 15:44:19.111730 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:19.111706    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"]
Jul 29 15:44:19.113955 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:44:19.113926    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d77471e_654b_491c_b01b_dcfb11ca097f.slice/crio-10661e15741280bb37c6cb43cd2b883b2ea8f0906ee16d277aeae3bd9340cb58 WatchSource:0}: Error finding container 10661e15741280bb37c6cb43cd2b883b2ea8f0906ee16d277aeae3bd9340cb58: Status 404 returned error can't find the container with id 10661e15741280bb37c6cb43cd2b883b2ea8f0906ee16d277aeae3bd9340cb58
Jul 29 15:44:19.115786 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:19.115764    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:44:19.420157 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:19.420117    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw" event={"ID":"2d77471e-654b-491c-b01b-dcfb11ca097f","Type":"ContainerStarted","Data":"10661e15741280bb37c6cb43cd2b883b2ea8f0906ee16d277aeae3bd9340cb58"}
Jul 29 15:44:23.447309 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:23.447270    2560 generic.go:356] "Generic (PLEG): container finished" podID="2d77471e-654b-491c-b01b-dcfb11ca097f" containerID="b13c60145fed086108967dcbadfd3c0ccac8ea031e7016c2d4023de35f0f45f7" exitCode=0
Jul 29 15:44:23.447717 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:23.447362    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw" event={"ID":"2d77471e-654b-491c-b01b-dcfb11ca097f","Type":"ContainerDied","Data":"b13c60145fed086108967dcbadfd3c0ccac8ea031e7016c2d4023de35f0f45f7"}
Jul 29 15:44:24.590066 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:24.590040    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"
Jul 29 15:44:24.702778 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:24.702685    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d77471e-654b-491c-b01b-dcfb11ca097f-maas-api-service-ca\") pod \"2d77471e-654b-491c-b01b-dcfb11ca097f\" (UID: \"2d77471e-654b-491c-b01b-dcfb11ca097f\") "
Jul 29 15:44:24.703039 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:24.703014    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d77471e-654b-491c-b01b-dcfb11ca097f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d77471e-654b-491c-b01b-dcfb11ca097f" (UID: "2d77471e-654b-491c-b01b-dcfb11ca097f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:44:24.804271 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:24.804231    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d77471e-654b-491c-b01b-dcfb11ca097f-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:44:25.459192 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:25.459151    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"
Jul 29 15:44:25.459370 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:25.459149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw" event={"ID":"2d77471e-654b-491c-b01b-dcfb11ca097f","Type":"ContainerDied","Data":"10661e15741280bb37c6cb43cd2b883b2ea8f0906ee16d277aeae3bd9340cb58"}
Jul 29 15:44:25.459370 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:25.459302    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10661e15741280bb37c6cb43cd2b883b2ea8f0906ee16d277aeae3bd9340cb58"
Jul 29 15:44:34.025815 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:34.025776    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"]
Jul 29 15:44:34.030654 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:34.030620    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93e16c15-62jbw"]
Jul 29 15:44:34.167363 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:34.167324    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d77471e-654b-491c-b01b-dcfb11ca097f" path="/var/lib/kubelet/pods/2d77471e-654b-491c-b01b-dcfb11ca097f/volumes"
Jul 29 15:44:37.690175 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:37.690136    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:44:48.282059 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:44:48.282018    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:45:00.140511 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.140467    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb"]
Jul 29 15:45:00.141004 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.140985    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d77471e-654b-491c-b01b-dcfb11ca097f" containerName="revoke-keys"
Jul 29 15:45:00.141051 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.141008    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d77471e-654b-491c-b01b-dcfb11ca097f" containerName="revoke-keys"
Jul 29 15:45:00.141137 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.141125    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d77471e-654b-491c-b01b-dcfb11ca097f" containerName="revoke-keys"
Jul 29 15:45:00.149443 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.149413    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"]
Jul 29 15:45:00.149589 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.149529    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb"
Jul 29 15:45:00.152610 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.152392    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 15:45:00.153518 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.153492    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"
Jul 29 15:45:00.154879 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.154858    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb"]
Jul 29 15:45:00.157603 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.157579    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"]
Jul 29 15:45:00.164496 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.164474    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb"
Jul 29 15:45:00.171891 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.171873    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"
Jul 29 15:45:00.318781 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.318747    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb"]
Jul 29 15:45:00.320449 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:45:00.320404    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f681ae_4558_4d3f_a30e_f268b3ad8517.slice/crio-ca4da636b5a3ba0748fcc4848c3e524404cee4fb032089692d28f4743d408416 WatchSource:0}: Error finding container ca4da636b5a3ba0748fcc4848c3e524404cee4fb032089692d28f4743d408416: Status 404 returned error can't find the container with id ca4da636b5a3ba0748fcc4848c3e524404cee4fb032089692d28f4743d408416
Jul 29 15:45:00.342864 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.342839    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"]
Jul 29 15:45:00.344904 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:45:00.344873    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4941de3c_2f9b_4768_b0d7_4db2d9a0a64e.slice/crio-938a5932bb3b3bb8d1928a7a4f2a6d66a4f96cc03a642d6fa97a795605bc12b0 WatchSource:0}: Error finding container 938a5932bb3b3bb8d1928a7a4f2a6d66a4f96cc03a642d6fa97a795605bc12b0: Status 404 returned error can't find the container with id 938a5932bb3b3bb8d1928a7a4f2a6d66a4f96cc03a642d6fa97a795605bc12b0
Jul 29 15:45:00.650491 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.650453    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb" event={"ID":"d2f681ae-4558-4d3f-a30e-f268b3ad8517","Type":"ContainerStarted","Data":"ca4da636b5a3ba0748fcc4848c3e524404cee4fb032089692d28f4743d408416"}
Jul 29 15:45:00.652264 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:00.652223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt" event={"ID":"4941de3c-2f9b-4768-b0d7-4db2d9a0a64e","Type":"ContainerStarted","Data":"938a5932bb3b3bb8d1928a7a4f2a6d66a4f96cc03a642d6fa97a795605bc12b0"}
Jul 29 15:45:01.666948 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:01.666914    2560 generic.go:356] "Generic (PLEG): container finished" podID="d2f681ae-4558-4d3f-a30e-f268b3ad8517" containerID="76dae8f00478f3f93ee3e80bbfe5a9d99f65328c7271054786c1d121848ff007" exitCode=0
Jul 29 15:45:01.667468 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:01.667004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb" event={"ID":"d2f681ae-4558-4d3f-a30e-f268b3ad8517","Type":"ContainerDied","Data":"76dae8f00478f3f93ee3e80bbfe5a9d99f65328c7271054786c1d121848ff007"}
Jul 29 15:45:01.668550 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:01.668524    2560 generic.go:356] "Generic (PLEG): container finished" podID="4941de3c-2f9b-4768-b0d7-4db2d9a0a64e" containerID="b73f7bb2aa7a3277f6d7c12ee1d15cb343f89985f921f5cb4f6f9cfafb2a6d5a" exitCode=0
Jul 29 15:45:01.668647 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:01.668556    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt" event={"ID":"4941de3c-2f9b-4768-b0d7-4db2d9a0a64e","Type":"ContainerDied","Data":"b73f7bb2aa7a3277f6d7c12ee1d15cb343f89985f921f5cb4f6f9cfafb2a6d5a"}
Jul 29 15:45:02.833434 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:02.833410    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"
Jul 29 15:45:02.836131 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:02.836113    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb"
Jul 29 15:45:03.376653 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.376617    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"]
Jul 29 15:45:03.377035 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.377016    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4941de3c-2f9b-4768-b0d7-4db2d9a0a64e" containerName="cleanup"
Jul 29 15:45:03.377125 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.377038    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4941de3c-2f9b-4768-b0d7-4db2d9a0a64e" containerName="cleanup"
Jul 29 15:45:03.377125 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.377067    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f681ae-4558-4d3f-a30e-f268b3ad8517" containerName="cleanup"
Jul 29 15:45:03.377125 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.377076    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f681ae-4558-4d3f-a30e-f268b3ad8517" containerName="cleanup"
Jul 29 15:45:03.377302 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.377201    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4941de3c-2f9b-4768-b0d7-4db2d9a0a64e" containerName="cleanup"
Jul 29 15:45:03.377302 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.377217    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2f681ae-4558-4d3f-a30e-f268b3ad8517" containerName="cleanup"
Jul 29 15:45:03.380313 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.380292    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"
Jul 29 15:45:03.382553 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.382536    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 15:45:03.387024 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.387000    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"]
Jul 29 15:45:03.440467 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.440430    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c349adb0-775f-4513-9d80-5418eaa65499-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx\" (UID: \"c349adb0-775f-4513-9d80-5418eaa65499\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"
Jul 29 15:45:03.541104 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.541058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c349adb0-775f-4513-9d80-5418eaa65499-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx\" (UID: \"c349adb0-775f-4513-9d80-5418eaa65499\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"
Jul 29 15:45:03.541769 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.541747    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c349adb0-775f-4513-9d80-5418eaa65499-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx\" (UID: \"c349adb0-775f-4513-9d80-5418eaa65499\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"
Jul 29 15:45:03.681006 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.680920    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb"
Jul 29 15:45:03.681006 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.680929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755665-vltfb" event={"ID":"d2f681ae-4558-4d3f-a30e-f268b3ad8517","Type":"ContainerDied","Data":"ca4da636b5a3ba0748fcc4848c3e524404cee4fb032089692d28f4743d408416"}
Jul 29 15:45:03.681006 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.680978    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca4da636b5a3ba0748fcc4848c3e524404cee4fb032089692d28f4743d408416"
Jul 29 15:45:03.682649 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.682628    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"
Jul 29 15:45:03.682772 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.682670    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt" event={"ID":"4941de3c-2f9b-4768-b0d7-4db2d9a0a64e","Type":"ContainerDied","Data":"938a5932bb3b3bb8d1928a7a4f2a6d66a4f96cc03a642d6fa97a795605bc12b0"}
Jul 29 15:45:03.682772 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.682720    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="938a5932bb3b3bb8d1928a7a4f2a6d66a4f96cc03a642d6fa97a795605bc12b0"
Jul 29 15:45:03.691013 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.690991    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"
Jul 29 15:45:03.829329 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:03.829153    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"]
Jul 29 15:45:03.832000 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:45:03.831971    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc349adb0_775f_4513_9d80_5418eaa65499.slice/crio-e17c7646ac73cbcb702d4fa0789c368bbf8141ec83cacf03fb35e8a984af8efe WatchSource:0}: Error finding container e17c7646ac73cbcb702d4fa0789c368bbf8141ec83cacf03fb35e8a984af8efe: Status 404 returned error can't find the container with id e17c7646ac73cbcb702d4fa0789c368bbf8141ec83cacf03fb35e8a984af8efe
Jul 29 15:45:04.690550 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:04.690519    2560 generic.go:356] "Generic (PLEG): container finished" podID="c349adb0-775f-4513-9d80-5418eaa65499" containerID="ea32a97466841e4368515d1aa000c1e1c7dd428482130f8c90d18e7134bd7b5e" exitCode=0
Jul 29 15:45:04.690917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:04.690607    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx" event={"ID":"c349adb0-775f-4513-9d80-5418eaa65499","Type":"ContainerDied","Data":"ea32a97466841e4368515d1aa000c1e1c7dd428482130f8c90d18e7134bd7b5e"}
Jul 29 15:45:04.690917 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:04.690658    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx" event={"ID":"c349adb0-775f-4513-9d80-5418eaa65499","Type":"ContainerStarted","Data":"e17c7646ac73cbcb702d4fa0789c368bbf8141ec83cacf03fb35e8a984af8efe"}
Jul 29 15:45:05.831496 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:05.831470    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"
Jul 29 15:45:05.861549 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:05.861514    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c349adb0-775f-4513-9d80-5418eaa65499-maas-api-service-ca\") pod \"c349adb0-775f-4513-9d80-5418eaa65499\" (UID: \"c349adb0-775f-4513-9d80-5418eaa65499\") "
Jul 29 15:45:05.861845 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:05.861824    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c349adb0-775f-4513-9d80-5418eaa65499-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c349adb0-775f-4513-9d80-5418eaa65499" (UID: "c349adb0-775f-4513-9d80-5418eaa65499"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:05.962407 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:05.962319    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c349adb0-775f-4513-9d80-5418eaa65499-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:45:06.704619 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:06.704579    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"
Jul 29 15:45:06.704619 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:06.704605    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx" event={"ID":"c349adb0-775f-4513-9d80-5418eaa65499","Type":"ContainerDied","Data":"e17c7646ac73cbcb702d4fa0789c368bbf8141ec83cacf03fb35e8a984af8efe"}
Jul 29 15:45:06.704840 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:06.704653    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e17c7646ac73cbcb702d4fa0789c368bbf8141ec83cacf03fb35e8a984af8efe"
Jul 29 15:45:09.727960 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:09.727921    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"]
Jul 29 15:45:09.731123 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:09.731098    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-adc0e91c-29755665-8wxpt"]
Jul 29 15:45:09.926741 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:09.926674    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"]
Jul 29 15:45:09.928906 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:09.928884    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-adc0e91c-5fmsx"]
Jul 29 15:45:10.166772 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:10.166737    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4941de3c-2f9b-4768-b0d7-4db2d9a0a64e" path="/var/lib/kubelet/pods/4941de3c-2f9b-4768-b0d7-4db2d9a0a64e/volumes"
Jul 29 15:45:10.167053 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:10.167040    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c349adb0-775f-4513-9d80-5418eaa65499" path="/var/lib/kubelet/pods/c349adb0-775f-4513-9d80-5418eaa65499/volumes"
Jul 29 15:45:14.088580 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:14.088546    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:45:24.789015 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:24.788921    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:45:39.227890 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.227854    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"]
Jul 29 15:45:39.228416 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.228248    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c349adb0-775f-4513-9d80-5418eaa65499" containerName="revoke-keys"
Jul 29 15:45:39.228416 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.228262    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c349adb0-775f-4513-9d80-5418eaa65499" containerName="revoke-keys"
Jul 29 15:45:39.228416 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.228342    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c349adb0-775f-4513-9d80-5418eaa65499" containerName="revoke-keys"
Jul 29 15:45:39.231312 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.231297    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"
Jul 29 15:45:39.233798 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.233779    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 15:45:39.233798 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.233785    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 15:45:39.242656 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.242622    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"]
Jul 29 15:45:39.359070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.359007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ca0a0e8-627c-4433-aa4c-59962c9a34f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-16426b01-j2ng8\" (UID: \"5ca0a0e8-627c-4433-aa4c-59962c9a34f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"
Jul 29 15:45:39.460037 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.459977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ca0a0e8-627c-4433-aa4c-59962c9a34f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-16426b01-j2ng8\" (UID: \"5ca0a0e8-627c-4433-aa4c-59962c9a34f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"
Jul 29 15:45:39.460628 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.460605    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ca0a0e8-627c-4433-aa4c-59962c9a34f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-16426b01-j2ng8\" (UID: \"5ca0a0e8-627c-4433-aa4c-59962c9a34f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"
Jul 29 15:45:39.545309 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.545228    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"
Jul 29 15:45:39.678840 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.678814    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"]
Jul 29 15:45:39.680497 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:45:39.680474    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca0a0e8_627c_4433_aa4c_59962c9a34f7.slice/crio-4ecf3bfac610cdaa0d149123068bee6423ac7148ec5bd557ba204869a49bfcf3 WatchSource:0}: Error finding container 4ecf3bfac610cdaa0d149123068bee6423ac7148ec5bd557ba204869a49bfcf3: Status 404 returned error can't find the container with id 4ecf3bfac610cdaa0d149123068bee6423ac7148ec5bd557ba204869a49bfcf3
Jul 29 15:45:39.880888 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:39.880859    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8" event={"ID":"5ca0a0e8-627c-4433-aa4c-59962c9a34f7","Type":"ContainerStarted","Data":"4ecf3bfac610cdaa0d149123068bee6423ac7148ec5bd557ba204869a49bfcf3"}
Jul 29 15:45:40.888126 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:40.888090    2560 generic.go:356] "Generic (PLEG): container finished" podID="5ca0a0e8-627c-4433-aa4c-59962c9a34f7" containerID="1cf8f4d5237b32bcfaa941a5c070a318c5e6b54d086efea15f893606735a3ebf" exitCode=0
Jul 29 15:45:40.888609 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:40.888131    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8" event={"ID":"5ca0a0e8-627c-4433-aa4c-59962c9a34f7","Type":"ContainerDied","Data":"1cf8f4d5237b32bcfaa941a5c070a318c5e6b54d086efea15f893606735a3ebf"}
Jul 29 15:45:42.024897 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:42.024870    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"
Jul 29 15:45:42.181359 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:42.181284    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ca0a0e8-627c-4433-aa4c-59962c9a34f7-maas-api-service-ca\") pod \"5ca0a0e8-627c-4433-aa4c-59962c9a34f7\" (UID: \"5ca0a0e8-627c-4433-aa4c-59962c9a34f7\") "
Jul 29 15:45:42.181642 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:42.181622    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ca0a0e8-627c-4433-aa4c-59962c9a34f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ca0a0e8-627c-4433-aa4c-59962c9a34f7" (UID: "5ca0a0e8-627c-4433-aa4c-59962c9a34f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:42.281939 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:42.281907    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ca0a0e8-627c-4433-aa4c-59962c9a34f7-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:45:42.899850 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:42.899823    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"
Jul 29 15:45:42.899850 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:42.899844    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8" event={"ID":"5ca0a0e8-627c-4433-aa4c-59962c9a34f7","Type":"ContainerDied","Data":"4ecf3bfac610cdaa0d149123068bee6423ac7148ec5bd557ba204869a49bfcf3"}
Jul 29 15:45:42.900075 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:42.899886    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ecf3bfac610cdaa0d149123068bee6423ac7148ec5bd557ba204869a49bfcf3"
Jul 29 15:45:44.328018 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:44.327969    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"]
Jul 29 15:45:44.330094 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:44.330073    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16426b01-j2ng8"]
Jul 29 15:45:46.166798 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:46.166764    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ca0a0e8-627c-4433-aa4c-59962c9a34f7" path="/var/lib/kubelet/pods/5ca0a0e8-627c-4433-aa4c-59962c9a34f7/volumes"
Jul 29 15:45:48.093199 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:48.093144    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:45:59.087233 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:45:59.087175    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:46:02.260097 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:02.260065    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:46:02.268529 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:02.268502    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:46:02.276426 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:02.276400    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:46:02.284645 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:02.284621    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:46:14.402947 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.402900    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"]
Jul 29 15:46:14.403533 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.403446    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ca0a0e8-627c-4433-aa4c-59962c9a34f7" containerName="revoke-keys"
Jul 29 15:46:14.403533 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.403467    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ca0a0e8-627c-4433-aa4c-59962c9a34f7" containerName="revoke-keys"
Jul 29 15:46:14.403652 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.403593    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ca0a0e8-627c-4433-aa4c-59962c9a34f7" containerName="revoke-keys"
Jul 29 15:46:14.406788 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.406763    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"
Jul 29 15:46:14.409133 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.409112    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 15:46:14.409284 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.409148    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 15:46:14.414298 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.414268    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"]
Jul 29 15:46:14.449621 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.449580    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0fe3df9-e801-4d6c-abd8-380b7b209d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1609e886-s4vqd\" (UID: \"c0fe3df9-e801-4d6c-abd8-380b7b209d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"
Jul 29 15:46:14.550818 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.550776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0fe3df9-e801-4d6c-abd8-380b7b209d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1609e886-s4vqd\" (UID: \"c0fe3df9-e801-4d6c-abd8-380b7b209d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"
Jul 29 15:46:14.551528 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.551505    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0fe3df9-e801-4d6c-abd8-380b7b209d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1609e886-s4vqd\" (UID: \"c0fe3df9-e801-4d6c-abd8-380b7b209d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"
Jul 29 15:46:14.719289 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.719160    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"
Jul 29 15:46:14.861393 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:14.861366    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"]
Jul 29 15:46:14.863702 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:46:14.863673    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fe3df9_e801_4d6c_abd8_380b7b209d52.slice/crio-15624c9efed95c176eba38156be29928e330ee6b6338fc92cea730027b6a1f10 WatchSource:0}: Error finding container 15624c9efed95c176eba38156be29928e330ee6b6338fc92cea730027b6a1f10: Status 404 returned error can't find the container with id 15624c9efed95c176eba38156be29928e330ee6b6338fc92cea730027b6a1f10
Jul 29 15:46:15.082166 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:15.082135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd" event={"ID":"c0fe3df9-e801-4d6c-abd8-380b7b209d52","Type":"ContainerStarted","Data":"15624c9efed95c176eba38156be29928e330ee6b6338fc92cea730027b6a1f10"}
Jul 29 15:46:16.089234 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:16.089202    2560 generic.go:356] "Generic (PLEG): container finished" podID="c0fe3df9-e801-4d6c-abd8-380b7b209d52" containerID="c11f0cc63d87659f7defbc6e18f342a307dfd24bdd6d42fc7d27beb94fd3d461" exitCode=0
Jul 29 15:46:16.089629 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:16.089288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd" event={"ID":"c0fe3df9-e801-4d6c-abd8-380b7b209d52","Type":"ContainerDied","Data":"c11f0cc63d87659f7defbc6e18f342a307dfd24bdd6d42fc7d27beb94fd3d461"}
Jul 29 15:46:17.228063 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:17.228031    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"
Jul 29 15:46:17.275923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:17.275893    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0fe3df9-e801-4d6c-abd8-380b7b209d52-maas-api-service-ca\") pod \"c0fe3df9-e801-4d6c-abd8-380b7b209d52\" (UID: \"c0fe3df9-e801-4d6c-abd8-380b7b209d52\") "
Jul 29 15:46:17.276326 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:17.276304    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0fe3df9-e801-4d6c-abd8-380b7b209d52-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0fe3df9-e801-4d6c-abd8-380b7b209d52" (UID: "c0fe3df9-e801-4d6c-abd8-380b7b209d52"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:46:17.377537 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:17.377448    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0fe3df9-e801-4d6c-abd8-380b7b209d52-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:46:18.101588 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:18.101552    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"
Jul 29 15:46:18.101853 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:18.101583    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd" event={"ID":"c0fe3df9-e801-4d6c-abd8-380b7b209d52","Type":"ContainerDied","Data":"15624c9efed95c176eba38156be29928e330ee6b6338fc92cea730027b6a1f10"}
Jul 29 15:46:18.101853 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:18.101669    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15624c9efed95c176eba38156be29928e330ee6b6338fc92cea730027b6a1f10"
Jul 29 15:46:20.489141 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:20.489103    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"]
Jul 29 15:46:20.492902 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:20.492869    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1609e886-s4vqd"]
Jul 29 15:46:22.165731 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:22.165692    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0fe3df9-e801-4d6c-abd8-380b7b209d52" path="/var/lib/kubelet/pods/c0fe3df9-e801-4d6c-abd8-380b7b209d52/volumes"
Jul 29 15:46:24.188107 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:24.188070    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:46:46.582352 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:46:46.582316    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:48:19.794246 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:48:19.794213    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:50:25.779137 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:50:25.779098    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:50:37.495379 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:50:37.495338    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:51:02.294277 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:02.294247    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:51:02.303278 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:02.303255    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:51:02.313135 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:02.313113    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:51:02.321112 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:02.321093    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:51:02.372555 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:02.372532    2560 scope.go:117] "RemoveContainer" containerID="b73f7bb2aa7a3277f6d7c12ee1d15cb343f89985f921f5cb4f6f9cfafb2a6d5a"
Jul 29 15:51:02.384214 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:02.384173    2560 scope.go:117] "RemoveContainer" containerID="b13c60145fed086108967dcbadfd3c0ccac8ea031e7016c2d4023de35f0f45f7"
Jul 29 15:51:04.554733 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.554695    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"]
Jul 29 15:51:04.555220 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.555198    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0fe3df9-e801-4d6c-abd8-380b7b209d52" containerName="revoke-keys"
Jul 29 15:51:04.555318 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.555222    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0fe3df9-e801-4d6c-abd8-380b7b209d52" containerName="revoke-keys"
Jul 29 15:51:04.555383 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.555324    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0fe3df9-e801-4d6c-abd8-380b7b209d52" containerName="revoke-keys"
Jul 29 15:51:04.559774 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.559753    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"
Jul 29 15:51:04.562161 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.562140    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 15:51:04.562277 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.562175    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 15:51:04.566263 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.566241    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"]
Jul 29 15:51:04.694922 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.694894    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd2735b-c72f-4f48-96cd-c08d7c96262b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z\" (UID: \"ffd2735b-c72f-4f48-96cd-c08d7c96262b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"
Jul 29 15:51:04.795424 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.795400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd2735b-c72f-4f48-96cd-c08d7c96262b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z\" (UID: \"ffd2735b-c72f-4f48-96cd-c08d7c96262b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"
Jul 29 15:51:04.795941 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.795923    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd2735b-c72f-4f48-96cd-c08d7c96262b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z\" (UID: \"ffd2735b-c72f-4f48-96cd-c08d7c96262b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"
Jul 29 15:51:04.871273 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:04.871255    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"
Jul 29 15:51:05.012475 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:05.012451    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"]
Jul 29 15:51:05.014013 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:51:05.013985    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd2735b_c72f_4f48_96cd_c08d7c96262b.slice/crio-6466d94cda36bc5f105d7d830f4055a56c4b36704d60ad3c97ff36ddd0808f1e WatchSource:0}: Error finding container 6466d94cda36bc5f105d7d830f4055a56c4b36704d60ad3c97ff36ddd0808f1e: Status 404 returned error can't find the container with id 6466d94cda36bc5f105d7d830f4055a56c4b36704d60ad3c97ff36ddd0808f1e
Jul 29 15:51:05.683123 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:05.683090    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z" event={"ID":"ffd2735b-c72f-4f48-96cd-c08d7c96262b","Type":"ContainerStarted","Data":"dc50e5fc034a64f3d80dab1e859bfc7b85b5a184aae93f0758da63863af008e5"}
Jul 29 15:51:05.683531 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:05.683142    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z" event={"ID":"ffd2735b-c72f-4f48-96cd-c08d7c96262b","Type":"ContainerStarted","Data":"6466d94cda36bc5f105d7d830f4055a56c4b36704d60ad3c97ff36ddd0808f1e"}
Jul 29 15:51:05.699333 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:05.699287    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z" podStartSLOduration=1.699272618 podStartE2EDuration="1.699272618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:51:05.698660785 +0000 UTC m=+2403.992597588" watchObservedRunningTime="2026-07-29 15:51:05.699272618 +0000 UTC m=+2403.993209420"
Jul 29 15:51:06.689340 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:06.689303    2560 generic.go:356] "Generic (PLEG): container finished" podID="ffd2735b-c72f-4f48-96cd-c08d7c96262b" containerID="dc50e5fc034a64f3d80dab1e859bfc7b85b5a184aae93f0758da63863af008e5" exitCode=0
Jul 29 15:51:06.689734 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:06.689385    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z" event={"ID":"ffd2735b-c72f-4f48-96cd-c08d7c96262b","Type":"ContainerDied","Data":"dc50e5fc034a64f3d80dab1e859bfc7b85b5a184aae93f0758da63863af008e5"}
Jul 29 15:51:07.824370 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:07.824344    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"
Jul 29 15:51:07.922440 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:07.922406    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd2735b-c72f-4f48-96cd-c08d7c96262b-maas-api-service-ca\") pod \"ffd2735b-c72f-4f48-96cd-c08d7c96262b\" (UID: \"ffd2735b-c72f-4f48-96cd-c08d7c96262b\") "
Jul 29 15:51:07.922761 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:07.922737    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd2735b-c72f-4f48-96cd-c08d7c96262b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ffd2735b-c72f-4f48-96cd-c08d7c96262b" (UID: "ffd2735b-c72f-4f48-96cd-c08d7c96262b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:51:08.023042 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:08.022956    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd2735b-c72f-4f48-96cd-c08d7c96262b-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:51:08.701151 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:08.701123    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"
Jul 29 15:51:08.701355 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:08.701121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z" event={"ID":"ffd2735b-c72f-4f48-96cd-c08d7c96262b","Type":"ContainerDied","Data":"6466d94cda36bc5f105d7d830f4055a56c4b36704d60ad3c97ff36ddd0808f1e"}
Jul 29 15:51:08.701355 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:08.701251    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6466d94cda36bc5f105d7d830f4055a56c4b36704d60ad3c97ff36ddd0808f1e"
Jul 29 15:51:15.317106 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:15.317074    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"]
Jul 29 15:51:15.321389 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:15.321363    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2a43d0f-jcl2z"]
Jul 29 15:51:16.166451 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:16.166416    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffd2735b-c72f-4f48-96cd-c08d7c96262b" path="/var/lib/kubelet/pods/ffd2735b-c72f-4f48-96cd-c08d7c96262b/volumes"
Jul 29 15:51:18.082199 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:18.082158    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:51:58.878334 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:51:58.878293    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:52:02.405954 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:52:02.405921    2560 scope.go:117] "RemoveContainer" containerID="1cf8f4d5237b32bcfaa941a5c070a318c5e6b54d086efea15f893606735a3ebf"
Jul 29 15:52:02.415026 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:52:02.415006    2560 scope.go:117] "RemoveContainer" containerID="ea32a97466841e4368515d1aa000c1e1c7dd428482130f8c90d18e7134bd7b5e"
Jul 29 15:53:02.439846 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:53:02.439758    2560 scope.go:117] "RemoveContainer" containerID="c11f0cc63d87659f7defbc6e18f342a307dfd24bdd6d42fc7d27beb94fd3d461"
Jul 29 15:53:44.200884 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:53:44.200833    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:53:56.192175 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:53:56.192134    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:54:20.375309 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.375273    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"]
Jul 29 15:54:20.375796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.375619    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffd2735b-c72f-4f48-96cd-c08d7c96262b" containerName="revoke-keys"
Jul 29 15:54:20.375796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.375630    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffd2735b-c72f-4f48-96cd-c08d7c96262b" containerName="revoke-keys"
Jul 29 15:54:20.375796 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.375692    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffd2735b-c72f-4f48-96cd-c08d7c96262b" containerName="revoke-keys"
Jul 29 15:54:20.378723 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.378703    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.383512 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.383487    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-87cc52d8\"/\"kube-root-ca.crt\""
Jul 29 15:54:20.383512 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.383499    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-87cc52d8\"/\"e2e-lifecycle-model-87cc52d8-kserve-self-signed-certs\""
Jul 29 15:54:20.383683 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.383510    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-87cc52d8\"/\"openshift-service-ca.crt\""
Jul 29 15:54:20.383683 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.383489    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-87cc52d8\"/\"default-dockercfg-prc5s\""
Jul 29 15:54:20.389724 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.389697    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"]
Jul 29 15:54:20.468423 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.468384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-home\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.468603 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.468438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq2cp\" (UniqueName: \"kubernetes.io/projected/d67ae362-f694-4b41-be3a-0f2a7fb3975b-kube-api-access-hq2cp\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.468603 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.468471    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tls-certs\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.468603 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.468499    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-model-cache\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.468603 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.468590    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-dshm\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.468750 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.468615    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tmp-dir\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.569514 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.569476    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-dshm\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.569514 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.569519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tmp-dir\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.569770 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.569560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-home\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.569770 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.569598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq2cp\" (UniqueName: \"kubernetes.io/projected/d67ae362-f694-4b41-be3a-0f2a7fb3975b-kube-api-access-hq2cp\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.569770 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.569625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tls-certs\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.569770 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.569662    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-model-cache\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.570006 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.569981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tmp-dir\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.570068 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.570017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-home\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.570068 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.570049    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-model-cache\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.571785 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.571767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-dshm\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.572256 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.572239    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tls-certs\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.579299 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.579271    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq2cp\" (UniqueName: \"kubernetes.io/projected/d67ae362-f694-4b41-be3a-0f2a7fb3975b-kube-api-access-hq2cp\") pod \"e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") " pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.689368 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.689268    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:20.826926 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:20.826899    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"]
Jul 29 15:54:20.828795 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:54:20.828763    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd67ae362_f694_4b41_be3a_0f2a7fb3975b.slice/crio-75b67f7351819fc4d45091e28b6bca843a0bef0f65b3719980269e1615655acd WatchSource:0}: Error finding container 75b67f7351819fc4d45091e28b6bca843a0bef0f65b3719980269e1615655acd: Status 404 returned error can't find the container with id 75b67f7351819fc4d45091e28b6bca843a0bef0f65b3719980269e1615655acd
Jul 29 15:54:21.730429 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:21.730392    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn" event={"ID":"d67ae362-f694-4b41-be3a-0f2a7fb3975b","Type":"ContainerStarted","Data":"d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962"}
Jul 29 15:54:21.730429 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:21.730434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn" event={"ID":"d67ae362-f694-4b41-be3a-0f2a7fb3975b","Type":"ContainerStarted","Data":"75b67f7351819fc4d45091e28b6bca843a0bef0f65b3719980269e1615655acd"}
Jul 29 15:54:21.751058 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:21.751000    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn" podStartSLOduration=1.750985187 podStartE2EDuration="1.750985187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:54:21.748396224 +0000 UTC m=+2600.042333027" watchObservedRunningTime="2026-07-29 15:54:21.750985187 +0000 UTC m=+2600.044921988"
Jul 29 15:54:28.887376 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:28.887288    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:54:30.689500 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:30.689464    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:30.689922 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:30.689515    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:30.702310 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:30.702280    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:30.788408 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:30.788370    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:54:37.987075 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:37.987035    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:54:56.185681 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:54:56.185633    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:55:06.485570 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:06.485529    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:55:06.497305 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:06.497274    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"]
Jul 29 15:55:21.516409 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.516360    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn" podUID="d67ae362-f694-4b41-be3a-0f2a7fb3975b" containerName="main" containerID="cri-o://d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962" gracePeriod=45
Jul 29 15:55:21.771412 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.771353    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:55:21.916017 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.915975    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-model-cache\") pod \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") "
Jul 29 15:55:21.916017 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916017    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tls-certs\") pod \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") "
Jul 29 15:55:21.916304 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916036    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-dshm\") pod \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") "
Jul 29 15:55:21.916304 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916072    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-home\") pod \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") "
Jul 29 15:55:21.916304 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916110    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tmp-dir\") pod \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") "
Jul 29 15:55:21.916304 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916135    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq2cp\" (UniqueName: \"kubernetes.io/projected/d67ae362-f694-4b41-be3a-0f2a7fb3975b-kube-api-access-hq2cp\") pod \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\" (UID: \"d67ae362-f694-4b41-be3a-0f2a7fb3975b\") "
Jul 29 15:55:21.916510 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916359    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-model-cache" (OuterVolumeSpecName: "model-cache") pod "d67ae362-f694-4b41-be3a-0f2a7fb3975b" (UID: "d67ae362-f694-4b41-be3a-0f2a7fb3975b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:55:21.916510 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-home" (OuterVolumeSpecName: "home") pod "d67ae362-f694-4b41-be3a-0f2a7fb3975b" (UID: "d67ae362-f694-4b41-be3a-0f2a7fb3975b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:55:21.916510 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.916448    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d67ae362-f694-4b41-be3a-0f2a7fb3975b" (UID: "d67ae362-f694-4b41-be3a-0f2a7fb3975b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:55:21.918311 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.918281    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-dshm" (OuterVolumeSpecName: "dshm") pod "d67ae362-f694-4b41-be3a-0f2a7fb3975b" (UID: "d67ae362-f694-4b41-be3a-0f2a7fb3975b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:55:21.918417 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.918363    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d67ae362-f694-4b41-be3a-0f2a7fb3975b" (UID: "d67ae362-f694-4b41-be3a-0f2a7fb3975b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:55:21.918459 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:21.918429    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d67ae362-f694-4b41-be3a-0f2a7fb3975b-kube-api-access-hq2cp" (OuterVolumeSpecName: "kube-api-access-hq2cp") pod "d67ae362-f694-4b41-be3a-0f2a7fb3975b" (UID: "d67ae362-f694-4b41-be3a-0f2a7fb3975b"). InnerVolumeSpecName "kube-api-access-hq2cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:55:22.017507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.017472    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:55:22.017507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.017502    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq2cp\" (UniqueName: \"kubernetes.io/projected/d67ae362-f694-4b41-be3a-0f2a7fb3975b-kube-api-access-hq2cp\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:55:22.017507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.017512    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:55:22.017757 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.017521    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67ae362-f694-4b41-be3a-0f2a7fb3975b-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:55:22.017757 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.017529    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:55:22.017757 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.017537    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d67ae362-f694-4b41-be3a-0f2a7fb3975b-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 15:55:22.046383 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.046307    2560 generic.go:356] "Generic (PLEG): container finished" podID="d67ae362-f694-4b41-be3a-0f2a7fb3975b" containerID="d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962" exitCode=0
Jul 29 15:55:22.046383 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.046349    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"
Jul 29 15:55:22.046552 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.046384    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn" event={"ID":"d67ae362-f694-4b41-be3a-0f2a7fb3975b","Type":"ContainerDied","Data":"d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962"}
Jul 29 15:55:22.046552 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.046428    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn" event={"ID":"d67ae362-f694-4b41-be3a-0f2a7fb3975b","Type":"ContainerDied","Data":"75b67f7351819fc4d45091e28b6bca843a0bef0f65b3719980269e1615655acd"}
Jul 29 15:55:22.046552 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.046445    2560 scope.go:117] "RemoveContainer" containerID="d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962"
Jul 29 15:55:22.056019 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.055992    2560 scope.go:117] "RemoveContainer" containerID="d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962"
Jul 29 15:55:22.056334 ip-10-0-136-199 kubenswrapper[2560]: E0729 15:55:22.056312    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962\": container with ID starting with d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962 not found: ID does not exist" containerID="d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962"
Jul 29 15:55:22.056419 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.056342    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962"} err="failed to get container status \"d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962\": rpc error: code = NotFound desc = could not find container \"d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962\": container with ID starting with d1947848f44f7a7b056f5089800f3ac77b67155a0657062b2ff65dde43008962 not found: ID does not exist"
Jul 29 15:55:22.069630 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.069603    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"]
Jul 29 15:55:22.074752 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.074726    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-87cc52d8/e2e-lifecycle-model-87cc52d8-kserve-7cbd48df6c-gfsrn"]
Jul 29 15:55:22.166296 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:22.166268    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d67ae362-f694-4b41-be3a-0f2a7fb3975b" path="/var/lib/kubelet/pods/d67ae362-f694-4b41-be3a-0f2a7fb3975b/volumes"
Jul 29 15:55:59.881313 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:55:59.881225    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:56:02.337286 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:56:02.337259    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:56:02.346071 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:56:02.346048    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:56:02.352949 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:56:02.352928    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 15:56:02.360934 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:56:02.360914    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 15:56:22.190629 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:56:22.190595    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:56:35.280171 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:56:35.280112    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:56:44.884494 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:56:44.884451    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:57:44.190415 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:57:44.190329    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:58:02.502457 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:02.502422    2560 scope.go:117] "RemoveContainer" containerID="dc50e5fc034a64f3d80dab1e859bfc7b85b5a184aae93f0758da63863af008e5"
Jul 29 15:58:13.699570 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:13.699528    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:58:43.876470 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.876434    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"]
Jul 29 15:58:43.877070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.876955    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d67ae362-f694-4b41-be3a-0f2a7fb3975b" containerName="main"
Jul 29 15:58:43.877070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.876976    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d67ae362-f694-4b41-be3a-0f2a7fb3975b" containerName="main"
Jul 29 15:58:43.877070 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.877068    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d67ae362-f694-4b41-be3a-0f2a7fb3975b" containerName="main"
Jul 29 15:58:43.880155 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.880131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:43.883374 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.883354    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9750ef\"/\"openshift-service-ca.crt\""
Jul 29 15:58:43.883526 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.883508    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9750ef\"/\"test-model-9750ef-kserve-self-signed-certs\""
Jul 29 15:58:43.883589 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.883531    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9750ef\"/\"default-dockercfg-7v4gp\""
Jul 29 15:58:43.883589 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.883516    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9750ef\"/\"kube-root-ca.crt\""
Jul 29 15:58:43.891058 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.891033    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"]
Jul 29 15:58:43.986495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.986459    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-home\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:43.986495 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.986500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxl8w\" (UniqueName: \"kubernetes.io/projected/4bdda744-d636-4700-b971-f9dad43498ff-kube-api-access-zxl8w\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:43.986721 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.986607    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-tmp-dir\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:43.986721 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.986638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdda744-d636-4700-b971-f9dad43498ff-tls-certs\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:43.986721 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.986694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-model-cache\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:43.986823 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:43.986733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-dshm\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.087593 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.087550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-dshm\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.087593 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.087599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-home\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.087870 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.087730    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxl8w\" (UniqueName: \"kubernetes.io/projected/4bdda744-d636-4700-b971-f9dad43498ff-kube-api-access-zxl8w\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.087870 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.087830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-tmp-dir\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.087870 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.087861    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdda744-d636-4700-b971-f9dad43498ff-tls-certs\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.088027 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.087922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-model-cache\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.088090 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.088030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-home\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.088234 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.088172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-tmp-dir\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.088340 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.088320    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-model-cache\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.089923 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.089896    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-dshm\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.090351 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.090331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdda744-d636-4700-b971-f9dad43498ff-tls-certs\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.098766 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.098739    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxl8w\" (UniqueName: \"kubernetes.io/projected/4bdda744-d636-4700-b971-f9dad43498ff-kube-api-access-zxl8w\") pod \"test-model-9750ef-kserve-69bb99f4b9-nkl9f\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") " pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.189674 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.189585    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:44.327232 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:44.327207    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"]
Jul 29 15:58:44.329157 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:58:44.329125    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bdda744_d636_4700_b971_f9dad43498ff.slice/crio-7f789dca9c04255ea8d19cd1738a36509e4d543b3336cd3479d4d25d0660ac75 WatchSource:0}: Error finding container 7f789dca9c04255ea8d19cd1738a36509e4d543b3336cd3479d4d25d0660ac75: Status 404 returned error can't find the container with id 7f789dca9c04255ea8d19cd1738a36509e4d543b3336cd3479d4d25d0660ac75
Jul 29 15:58:45.137511 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:45.137472    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f" event={"ID":"4bdda744-d636-4700-b971-f9dad43498ff","Type":"ContainerStarted","Data":"33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef"}
Jul 29 15:58:45.137976 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:45.137519    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f" event={"ID":"4bdda744-d636-4700-b971-f9dad43498ff","Type":"ContainerStarted","Data":"7f789dca9c04255ea8d19cd1738a36509e4d543b3336cd3479d4d25d0660ac75"}
Jul 29 15:58:45.158050 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:45.157996    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f" podStartSLOduration=2.157981829 podStartE2EDuration="2.157981829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:58:45.156695137 +0000 UTC m=+2863.450631940" watchObservedRunningTime="2026-07-29 15:58:45.157981829 +0000 UTC m=+2863.451918632"
Jul 29 15:58:48.691478 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:48.691436    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:58:54.190516 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:54.190413    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:54.190516 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:54.190481    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:54.203499 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:54.203465    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:58:55.208732 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:58:55.208700    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 15:59:29.687046 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.687008    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"]
Jul 29 15:59:29.690646 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.690625    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.693962 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.693934    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-dfc210\"/\"default-dockercfg-whwt9\""
Jul 29 15:59:29.694082 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.693960    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-dfc210\"/\"kube-root-ca.crt\""
Jul 29 15:59:29.694082 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.694000    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-dfc210\"/\"openshift-service-ca.crt\""
Jul 29 15:59:29.694082 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.693933    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-dfc210\"/\"test-model-dfc210-kserve-self-signed-certs\""
Jul 29 15:59:29.699977 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.699952    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"]
Jul 29 15:59:29.786478 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.786444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-dshm\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.786654 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.786490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tmp-dir\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.786654 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.786559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-home\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.786654 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.786592    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-model-cache\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.786654 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.786623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tls-certs\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.786791 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.786742    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhvv9\" (UniqueName: \"kubernetes.io/projected/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-kube-api-access-bhvv9\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.887854 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.887749    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tls-certs\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888029 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.887918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhvv9\" (UniqueName: \"kubernetes.io/projected/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-kube-api-access-bhvv9\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888029 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.887959    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-dshm\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888029 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.887991    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tmp-dir\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888167 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.888031    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-home\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888167 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.888056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-model-cache\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888410 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.888390    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-model-cache\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888504 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.888482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tmp-dir\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.888563 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.888490    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-home\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.890208 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.890165    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-dshm\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.890507 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.890474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tls-certs\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:29.897360 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:29.897337    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhvv9\" (UniqueName: \"kubernetes.io/projected/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-kube-api-access-bhvv9\") pod \"test-model-dfc210-kserve-7d4f47f78f-f4p2m\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") " pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:30.002623 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:30.002524    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:30.143924 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:30.143898    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"]
Jul 29 15:59:30.145915 ip-10-0-136-199 kubenswrapper[2560]: W0729 15:59:30.145879    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83a41149_e1d3_4992_9a47_3ddfb7ee2a7d.slice/crio-24eb13cb04d8b0a630521249fe0e53cf049f2b130bee522417d48294fb8d4d3a WatchSource:0}: Error finding container 24eb13cb04d8b0a630521249fe0e53cf049f2b130bee522417d48294fb8d4d3a: Status 404 returned error can't find the container with id 24eb13cb04d8b0a630521249fe0e53cf049f2b130bee522417d48294fb8d4d3a
Jul 29 15:59:30.395420 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:30.395383    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m" event={"ID":"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d","Type":"ContainerStarted","Data":"ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac"}
Jul 29 15:59:30.395614 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:30.395435    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m" event={"ID":"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d","Type":"ContainerStarted","Data":"24eb13cb04d8b0a630521249fe0e53cf049f2b130bee522417d48294fb8d4d3a"}
Jul 29 15:59:30.415372 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:30.415306    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m" podStartSLOduration=1.415290697 podStartE2EDuration="1.415290697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:59:30.413238212 +0000 UTC m=+2908.707175015" watchObservedRunningTime="2026-07-29 15:59:30.415290697 +0000 UTC m=+2908.709227565"
Jul 29 15:59:35.398069 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:35.398023    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 15:59:40.002702 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:40.002659    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:40.003128 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:40.002721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:40.015277 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:40.015252    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 15:59:40.461260 ip-10-0-136-199 kubenswrapper[2560]: I0729 15:59:40.461233    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 16:00:14.259010 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:14.258966    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"]
Jul 29 16:00:15.274670 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:15.274631    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"]
Jul 29 16:00:17.362246 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:17.362206    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:00:29.092309 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.092214    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:00:29.284335 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.284286    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f" podUID="4bdda744-d636-4700-b971-f9dad43498ff" containerName="main" containerID="cri-o://33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef" gracePeriod=45
Jul 29 16:00:29.536513 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.536484    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 16:00:29.608367 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608331    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-model-cache\") pod \"4bdda744-d636-4700-b971-f9dad43498ff\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") "
Jul 29 16:00:29.608574 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608372    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-home\") pod \"4bdda744-d636-4700-b971-f9dad43498ff\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") "
Jul 29 16:00:29.608574 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608405    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdda744-d636-4700-b971-f9dad43498ff-tls-certs\") pod \"4bdda744-d636-4700-b971-f9dad43498ff\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") "
Jul 29 16:00:29.608574 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608434    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-dshm\") pod \"4bdda744-d636-4700-b971-f9dad43498ff\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") "
Jul 29 16:00:29.608574 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608505    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxl8w\" (UniqueName: \"kubernetes.io/projected/4bdda744-d636-4700-b971-f9dad43498ff-kube-api-access-zxl8w\") pod \"4bdda744-d636-4700-b971-f9dad43498ff\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") "
Jul 29 16:00:29.608574 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608533    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-tmp-dir\") pod \"4bdda744-d636-4700-b971-f9dad43498ff\" (UID: \"4bdda744-d636-4700-b971-f9dad43498ff\") "
Jul 29 16:00:29.608843 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608660    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "4bdda744-d636-4700-b971-f9dad43498ff" (UID: "4bdda744-d636-4700-b971-f9dad43498ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:29.608843 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608679    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-home" (OuterVolumeSpecName: "home") pod "4bdda744-d636-4700-b971-f9dad43498ff" (UID: "4bdda744-d636-4700-b971-f9dad43498ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:29.608940 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608872    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:29.608940 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608892    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:29.608940 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.608888    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4bdda744-d636-4700-b971-f9dad43498ff" (UID: "4bdda744-d636-4700-b971-f9dad43498ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:29.610828 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.610807    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-dshm" (OuterVolumeSpecName: "dshm") pod "4bdda744-d636-4700-b971-f9dad43498ff" (UID: "4bdda744-d636-4700-b971-f9dad43498ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:29.610917 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.610893    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bdda744-d636-4700-b971-f9dad43498ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4bdda744-d636-4700-b971-f9dad43498ff" (UID: "4bdda744-d636-4700-b971-f9dad43498ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:00:29.610955 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.610893    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bdda744-d636-4700-b971-f9dad43498ff-kube-api-access-zxl8w" (OuterVolumeSpecName: "kube-api-access-zxl8w") pod "4bdda744-d636-4700-b971-f9dad43498ff" (UID: "4bdda744-d636-4700-b971-f9dad43498ff"). InnerVolumeSpecName "kube-api-access-zxl8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:29.709645 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.709541    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bdda744-d636-4700-b971-f9dad43498ff-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:29.709645 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.709576    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:29.709645 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.709589    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxl8w\" (UniqueName: \"kubernetes.io/projected/4bdda744-d636-4700-b971-f9dad43498ff-kube-api-access-zxl8w\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:29.709645 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.709602    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdda744-d636-4700-b971-f9dad43498ff-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:29.722370 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.722340    2560 generic.go:356] "Generic (PLEG): container finished" podID="4bdda744-d636-4700-b971-f9dad43498ff" containerID="33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef" exitCode=0
Jul 29 16:00:29.722544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.722379    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"
Jul 29 16:00:29.722544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.722431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f" event={"ID":"4bdda744-d636-4700-b971-f9dad43498ff","Type":"ContainerDied","Data":"33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef"}
Jul 29 16:00:29.722544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.722476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f" event={"ID":"4bdda744-d636-4700-b971-f9dad43498ff","Type":"ContainerDied","Data":"7f789dca9c04255ea8d19cd1738a36509e4d543b3336cd3479d4d25d0660ac75"}
Jul 29 16:00:29.722544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.722500    2560 scope.go:117] "RemoveContainer" containerID="33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef"
Jul 29 16:00:29.733217 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.733173    2560 scope.go:117] "RemoveContainer" containerID="33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef"
Jul 29 16:00:29.733513 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:00:29.733494    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef\": container with ID starting with 33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef not found: ID does not exist" containerID="33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef"
Jul 29 16:00:29.733577 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.733524    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef"} err="failed to get container status \"33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef\": rpc error: code = NotFound desc = could not find container \"33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef\": container with ID starting with 33795f0190a3093323e668308b2d4cf739a1d410b184a3545078836a22a018ef not found: ID does not exist"
Jul 29 16:00:29.750878 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.749099    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"]
Jul 29 16:00:29.755559 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:29.755520    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9750ef/test-model-9750ef-kserve-69bb99f4b9-nkl9f"]
Jul 29 16:00:30.167022 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.166990    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bdda744-d636-4700-b971-f9dad43498ff" path="/var/lib/kubelet/pods/4bdda744-d636-4700-b971-f9dad43498ff/volumes"
Jul 29 16:00:30.293642 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.293606    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m" podUID="83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" containerName="main" containerID="cri-o://ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac" gracePeriod=45
Jul 29 16:00:30.451312 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.451219    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m" podUID="83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": dial tcp 10.132.0.60:8000: connect: connection refused"
Jul 29 16:00:30.549598 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.549572    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 16:00:30.617441 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617409    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tmp-dir\") pod \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") "
Jul 29 16:00:30.617441 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617445    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-home\") pod \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") "
Jul 29 16:00:30.617709 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617470    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-dshm\") pod \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") "
Jul 29 16:00:30.617709 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617494    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhvv9\" (UniqueName: \"kubernetes.io/projected/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-kube-api-access-bhvv9\") pod \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") "
Jul 29 16:00:30.617709 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617525    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tls-certs\") pod \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") "
Jul 29 16:00:30.617709 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-model-cache\") pod \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\" (UID: \"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d\") "
Jul 29 16:00:30.617907 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617764    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-home" (OuterVolumeSpecName: "home") pod "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" (UID: "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:30.617907 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617774    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" (UID: "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:30.617998 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.617916    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" (UID: "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:30.619789 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.619763    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-dshm" (OuterVolumeSpecName: "dshm") pod "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" (UID: "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:00:30.619900 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.619816    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" (UID: "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:00:30.619900 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.619864    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-kube-api-access-bhvv9" (OuterVolumeSpecName: "kube-api-access-bhvv9") pod "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" (UID: "83a41149-e1d3-4992-9a47-3ddfb7ee2a7d"). InnerVolumeSpecName "kube-api-access-bhvv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:30.718465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.718365    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:30.718465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.718401    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:30.718465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.718410    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:30.718465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.718419    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhvv9\" (UniqueName: \"kubernetes.io/projected/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-kube-api-access-bhvv9\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:30.718465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.718430    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:30.718465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.718438    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:30.730080 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.730052    2560 generic.go:356] "Generic (PLEG): container finished" podID="83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" containerID="ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac" exitCode=0
Jul 29 16:00:30.730262 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.730098    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"
Jul 29 16:00:30.730262 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.730145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m" event={"ID":"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d","Type":"ContainerDied","Data":"ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac"}
Jul 29 16:00:30.730262 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.730206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m" event={"ID":"83a41149-e1d3-4992-9a47-3ddfb7ee2a7d","Type":"ContainerDied","Data":"24eb13cb04d8b0a630521249fe0e53cf049f2b130bee522417d48294fb8d4d3a"}
Jul 29 16:00:30.730262 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.730226    2560 scope.go:117] "RemoveContainer" containerID="ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac"
Jul 29 16:00:30.739949 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.739928    2560 scope.go:117] "RemoveContainer" containerID="ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac"
Jul 29 16:00:30.740252 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:00:30.740231    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac\": container with ID starting with ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac not found: ID does not exist" containerID="ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac"
Jul 29 16:00:30.740315 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.740261    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac"} err="failed to get container status \"ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac\": rpc error: code = NotFound desc = could not find container \"ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac\": container with ID starting with ceacd62f5c78b1efc79164cc8021d23b061b154f91d8cb5936ab0fae28b2dfac not found: ID does not exist"
Jul 29 16:00:30.753342 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.753317    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"]
Jul 29 16:00:30.757947 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:30.757924    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-dfc210/test-model-dfc210-kserve-7d4f47f78f-f4p2m"]
Jul 29 16:00:32.167495 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:32.167463    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" path="/var/lib/kubelet/pods/83a41149-e1d3-4992-9a47-3ddfb7ee2a7d/volumes"
Jul 29 16:00:33.834784 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.834749    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"]
Jul 29 16:00:33.835231 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.835084    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bdda744-d636-4700-b971-f9dad43498ff" containerName="main"
Jul 29 16:00:33.835231 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.835095    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bdda744-d636-4700-b971-f9dad43498ff" containerName="main"
Jul 29 16:00:33.835231 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.835117    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" containerName="main"
Jul 29 16:00:33.835231 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.835123    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" containerName="main"
Jul 29 16:00:33.835231 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.835205    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bdda744-d636-4700-b971-f9dad43498ff" containerName="main"
Jul 29 16:00:33.835231 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.835217    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a41149-e1d3-4992-9a47-3ddfb7ee2a7d" containerName="main"
Jul 29 16:00:33.839686 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.839667    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"
Jul 29 16:00:33.842652 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.842631    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 16:00:33.842761 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.842716    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:00:33.847079 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.847022    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"]
Jul 29 16:00:33.944522 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:33.944487    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj\" (UID: \"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"
Jul 29 16:00:34.045718 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:34.045678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj\" (UID: \"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"
Jul 29 16:00:34.046370 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:34.046350    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj\" (UID: \"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"
Jul 29 16:00:34.152298 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:34.152258    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"
Jul 29 16:00:34.290921 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:34.290895    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"]
Jul 29 16:00:34.292853 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:00:34.292827    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a6bdfdb_a5fc_4cd5_93bd_5ad0cce8f0d7.slice/crio-04891e565316c737272b95cc3ea81e62c2900cb8ac04ae20fc6dde9207836cd5 WatchSource:0}: Error finding container 04891e565316c737272b95cc3ea81e62c2900cb8ac04ae20fc6dde9207836cd5: Status 404 returned error can't find the container with id 04891e565316c737272b95cc3ea81e62c2900cb8ac04ae20fc6dde9207836cd5
Jul 29 16:00:34.754117 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:34.754026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj" event={"ID":"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7","Type":"ContainerStarted","Data":"c7e347ac185a73329414b1e8cfcb5ce71b6c9e08ee64a995c765c76ca9212e2b"}
Jul 29 16:00:34.754117 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:34.754083    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj" event={"ID":"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7","Type":"ContainerStarted","Data":"04891e565316c737272b95cc3ea81e62c2900cb8ac04ae20fc6dde9207836cd5"}
Jul 29 16:00:34.776762 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:34.776707    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj" podStartSLOduration=1.776689333 podStartE2EDuration="1.776689333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:00:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:00:34.775061723 +0000 UTC m=+2973.068998527" watchObservedRunningTime="2026-07-29 16:00:34.776689333 +0000 UTC m=+2973.070626134"
Jul 29 16:00:35.760781 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:35.760739    2560 generic.go:356] "Generic (PLEG): container finished" podID="6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7" containerID="c7e347ac185a73329414b1e8cfcb5ce71b6c9e08ee64a995c765c76ca9212e2b" exitCode=0
Jul 29 16:00:35.761257 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:35.760817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj" event={"ID":"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7","Type":"ContainerDied","Data":"c7e347ac185a73329414b1e8cfcb5ce71b6c9e08ee64a995c765c76ca9212e2b"}
Jul 29 16:00:36.904781 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:36.904749    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"
Jul 29 16:00:36.970429 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:36.970397    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7-maas-api-service-ca\") pod \"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7\" (UID: \"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7\") "
Jul 29 16:00:36.970780 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:36.970756    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7" (UID: "6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:00:37.071628 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:37.071543    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:00:37.773764 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:37.773730    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"
Jul 29 16:00:37.773764 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:37.773754    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj" event={"ID":"6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7","Type":"ContainerDied","Data":"04891e565316c737272b95cc3ea81e62c2900cb8ac04ae20fc6dde9207836cd5"}
Jul 29 16:00:37.774003 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:37.773806    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04891e565316c737272b95cc3ea81e62c2900cb8ac04ae20fc6dde9207836cd5"
Jul 29 16:00:39.269244 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:39.269162    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"]
Jul 29 16:00:39.274488 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:39.274460    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dfc210-ttlxj"]
Jul 29 16:00:40.166315 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:40.166280    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7" path="/var/lib/kubelet/pods/6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7/volumes"
Jul 29 16:00:41.182943 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:41.182906    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:00:56.683928 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:00:56.683891    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:01:02.371692 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:02.371660    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:01:02.386605 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:02.386577    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:01:02.394752 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:02.394727    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:01:02.402345 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:02.402321    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:01:20.843372 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.843331    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"]
Jul 29 16:01:20.843872 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.843687    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7" containerName="revoke-keys"
Jul 29 16:01:20.843872 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.843698    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7" containerName="revoke-keys"
Jul 29 16:01:20.843872 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.843755    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a6bdfdb-a5fc-4cd5-93bd-5ad0cce8f0d7" containerName="revoke-keys"
Jul 29 16:01:20.846904 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.846876    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:20.849453 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.849431    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e5a566\""
Jul 29 16:01:20.849631 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.849613    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e5a566-dockercfg-6pgkg\""
Jul 29 16:01:20.859504 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.859479    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"]
Jul 29 16:01:20.957343 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.957305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df77936-12af-4f06-9a95-2f3601466ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:20.957541 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:20.957441    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghg8b\" (UniqueName: \"kubernetes.io/projected/2df77936-12af-4f06-9a95-2f3601466ac4-kube-api-access-ghg8b\") pod \"payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:21.058887 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:21.058841    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghg8b\" (UniqueName: \"kubernetes.io/projected/2df77936-12af-4f06-9a95-2f3601466ac4-kube-api-access-ghg8b\") pod \"payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:21.059094 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:21.058940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df77936-12af-4f06-9a95-2f3601466ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:21.059715 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:21.059693    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df77936-12af-4f06-9a95-2f3601466ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:21.068290 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:21.068260    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghg8b\" (UniqueName: \"kubernetes.io/projected/2df77936-12af-4f06-9a95-2f3601466ac4-kube-api-access-ghg8b\") pod \"payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:21.156868 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:21.156776    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:21.501910 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:21.501881    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"]
Jul 29 16:01:21.502875 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:01:21.502851    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df77936_12af_4f06_9a95_2f3601466ac4.slice/crio-275a2f4dabc6297cc878940dc73db0189f2259ae29922a3d17db7d9c8b0c98ca WatchSource:0}: Error finding container 275a2f4dabc6297cc878940dc73db0189f2259ae29922a3d17db7d9c8b0c98ca: Status 404 returned error can't find the container with id 275a2f4dabc6297cc878940dc73db0189f2259ae29922a3d17db7d9c8b0c98ca
Jul 29 16:01:21.504701 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:21.504684    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:01:22.016362 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:22.016329    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm" event={"ID":"2df77936-12af-4f06-9a95-2f3601466ac4","Type":"ContainerStarted","Data":"275a2f4dabc6297cc878940dc73db0189f2259ae29922a3d17db7d9c8b0c98ca"}
Jul 29 16:01:23.398074 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:23.398033    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:01:24.030686 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:24.030645    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm" event={"ID":"2df77936-12af-4f06-9a95-2f3601466ac4","Type":"ContainerStarted","Data":"179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1"}
Jul 29 16:01:24.030880 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:24.030774    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:24.053584 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:24.053530    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm" podStartSLOduration=2.380725042 podStartE2EDuration="4.053512378s" totalImagesPullingTime="1.672787336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:01:24.052463947 +0000 UTC m=+3022.346400750" watchObservedRunningTime="2026-07-29 16:01:24.053512378 +0000 UTC m=+3022.347449252"
Jul 29 16:01:35.037701 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:35.037668    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:01:47.718474 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.718426    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"]
Jul 29 16:01:47.722689 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.722664    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"
Jul 29 16:01:47.724970 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.724944    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 16:01:47.725242 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.725051    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:01:47.729795 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.729768    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"]
Jul 29 16:01:47.780860 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.780818    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a096fa42-0296-4d62-8a4b-fed93adf0b45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-31add0-s5r6t\" (UID: \"a096fa42-0296-4d62-8a4b-fed93adf0b45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"
Jul 29 16:01:47.882076 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.882034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a096fa42-0296-4d62-8a4b-fed93adf0b45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-31add0-s5r6t\" (UID: \"a096fa42-0296-4d62-8a4b-fed93adf0b45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"
Jul 29 16:01:47.882956 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:47.882931    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a096fa42-0296-4d62-8a4b-fed93adf0b45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-31add0-s5r6t\" (UID: \"a096fa42-0296-4d62-8a4b-fed93adf0b45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"
Jul 29 16:01:48.035738 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:48.035647    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"
Jul 29 16:01:48.176816 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:48.176788    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"]
Jul 29 16:01:48.178818 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:01:48.178788    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda096fa42_0296_4d62_8a4b_fed93adf0b45.slice/crio-7f8aeff5286464675601d23c18b2ca414d25185d146e51f1e75897fc121d7c79 WatchSource:0}: Error finding container 7f8aeff5286464675601d23c18b2ca414d25185d146e51f1e75897fc121d7c79: Status 404 returned error can't find the container with id 7f8aeff5286464675601d23c18b2ca414d25185d146e51f1e75897fc121d7c79
Jul 29 16:01:49.168017 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:49.167983    2560 generic.go:356] "Generic (PLEG): container finished" podID="a096fa42-0296-4d62-8a4b-fed93adf0b45" containerID="5b5b92ed7ac86b0b6753cb85ad2153746f4e98740d0c3538c4c4a4dbd785dc9f" exitCode=0
Jul 29 16:01:49.168475 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:49.168051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t" event={"ID":"a096fa42-0296-4d62-8a4b-fed93adf0b45","Type":"ContainerDied","Data":"5b5b92ed7ac86b0b6753cb85ad2153746f4e98740d0c3538c4c4a4dbd785dc9f"}
Jul 29 16:01:49.168475 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:49.168088    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t" event={"ID":"a096fa42-0296-4d62-8a4b-fed93adf0b45","Type":"ContainerStarted","Data":"7f8aeff5286464675601d23c18b2ca414d25185d146e51f1e75897fc121d7c79"}
Jul 29 16:01:50.306771 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:50.306742    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"
Jul 29 16:01:50.404052 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:50.403994    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a096fa42-0296-4d62-8a4b-fed93adf0b45-maas-api-service-ca\") pod \"a096fa42-0296-4d62-8a4b-fed93adf0b45\" (UID: \"a096fa42-0296-4d62-8a4b-fed93adf0b45\") "
Jul 29 16:01:50.404429 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:50.404405    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a096fa42-0296-4d62-8a4b-fed93adf0b45-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a096fa42-0296-4d62-8a4b-fed93adf0b45" (UID: "a096fa42-0296-4d62-8a4b-fed93adf0b45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:01:50.505793 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:50.505685    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a096fa42-0296-4d62-8a4b-fed93adf0b45-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:01:51.179706 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:51.179676    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"
Jul 29 16:01:51.179706 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:51.179701    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t" event={"ID":"a096fa42-0296-4d62-8a4b-fed93adf0b45","Type":"ContainerDied","Data":"7f8aeff5286464675601d23c18b2ca414d25185d146e51f1e75897fc121d7c79"}
Jul 29 16:01:51.179965 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:01:51.179747    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f8aeff5286464675601d23c18b2ca414d25185d146e51f1e75897fc121d7c79"
Jul 29 16:02:03.564852 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:03.564765    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"]
Jul 29 16:02:03.568855 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:03.568828    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-31add0-s5r6t"]
Jul 29 16:02:04.167037 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:04.166995    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a096fa42-0296-4d62-8a4b-fed93adf0b45" path="/var/lib/kubelet/pods/a096fa42-0296-4d62-8a4b-fed93adf0b45/volumes"
Jul 29 16:02:06.481109 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:06.481070    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:02:26.395804 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.395733    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"]
Jul 29 16:02:26.396749 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.396710    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm" podUID="2df77936-12af-4f06-9a95-2f3601466ac4" containerName="payload-pre-processing" containerID="cri-o://179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1" gracePeriod=30
Jul 29 16:02:26.653408 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.653334    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:02:26.728057 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.728021    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df77936-12af-4f06-9a95-2f3601466ac4-plugins-config-volume\") pod \"2df77936-12af-4f06-9a95-2f3601466ac4\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") "
Jul 29 16:02:26.728279 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.728090    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghg8b\" (UniqueName: \"kubernetes.io/projected/2df77936-12af-4f06-9a95-2f3601466ac4-kube-api-access-ghg8b\") pod \"2df77936-12af-4f06-9a95-2f3601466ac4\" (UID: \"2df77936-12af-4f06-9a95-2f3601466ac4\") "
Jul 29 16:02:26.728431 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.728407    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2df77936-12af-4f06-9a95-2f3601466ac4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2df77936-12af-4f06-9a95-2f3601466ac4" (UID: "2df77936-12af-4f06-9a95-2f3601466ac4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:02:26.730409 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.730381    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df77936-12af-4f06-9a95-2f3601466ac4-kube-api-access-ghg8b" (OuterVolumeSpecName: "kube-api-access-ghg8b") pod "2df77936-12af-4f06-9a95-2f3601466ac4" (UID: "2df77936-12af-4f06-9a95-2f3601466ac4"). InnerVolumeSpecName "kube-api-access-ghg8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:02:26.829563 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.829512    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df77936-12af-4f06-9a95-2f3601466ac4-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:02:26.829563 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:26.829551    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghg8b\" (UniqueName: \"kubernetes.io/projected/2df77936-12af-4f06-9a95-2f3601466ac4-kube-api-access-ghg8b\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:02:27.388759 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.388720    2560 generic.go:356] "Generic (PLEG): container finished" podID="2df77936-12af-4f06-9a95-2f3601466ac4" containerID="179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1" exitCode=0
Jul 29 16:02:27.388941 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.388767    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"
Jul 29 16:02:27.388941 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.388808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm" event={"ID":"2df77936-12af-4f06-9a95-2f3601466ac4","Type":"ContainerDied","Data":"179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1"}
Jul 29 16:02:27.388941 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.388865    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm" event={"ID":"2df77936-12af-4f06-9a95-2f3601466ac4","Type":"ContainerDied","Data":"275a2f4dabc6297cc878940dc73db0189f2259ae29922a3d17db7d9c8b0c98ca"}
Jul 29 16:02:27.388941 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.388890    2560 scope.go:117] "RemoveContainer" containerID="179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1"
Jul 29 16:02:27.398664 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.398476    2560 scope.go:117] "RemoveContainer" containerID="179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1"
Jul 29 16:02:27.398922 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:02:27.398783    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1\": container with ID starting with 179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1 not found: ID does not exist" containerID="179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1"
Jul 29 16:02:27.398922 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.398805    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1"} err="failed to get container status \"179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1\": rpc error: code = NotFound desc = could not find container \"179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1\": container with ID starting with 179f85f597404d10e83e107cb400c9061182d6ffa01bd36b2947381e0af1bbf1 not found: ID does not exist"
Jul 29 16:02:27.412847 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.412811    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"]
Jul 29 16:02:27.416918 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:27.416888    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e5a566-76754888dd-2jcsm"]
Jul 29 16:02:28.172591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:28.172550    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2df77936-12af-4f06-9a95-2f3601466ac4" path="/var/lib/kubelet/pods/2df77936-12af-4f06-9a95-2f3601466ac4/volumes"
Jul 29 16:02:31.902461 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:31.902424    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:02:38.878195 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.878145    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"]
Jul 29 16:02:38.878685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.878519    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df77936-12af-4f06-9a95-2f3601466ac4" containerName="payload-pre-processing"
Jul 29 16:02:38.878685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.878530    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df77936-12af-4f06-9a95-2f3601466ac4" containerName="payload-pre-processing"
Jul 29 16:02:38.878685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.878545    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a096fa42-0296-4d62-8a4b-fed93adf0b45" containerName="revoke-keys"
Jul 29 16:02:38.878685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.878552    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a096fa42-0296-4d62-8a4b-fed93adf0b45" containerName="revoke-keys"
Jul 29 16:02:38.878685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.878612    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df77936-12af-4f06-9a95-2f3601466ac4" containerName="payload-pre-processing"
Jul 29 16:02:38.878685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.878622    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a096fa42-0296-4d62-8a4b-fed93adf0b45" containerName="revoke-keys"
Jul 29 16:02:38.883020 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.883003    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:38.885625 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.885599    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-02add2-dockercfg-2d9rx\""
Jul 29 16:02:38.885757 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.885640    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-02add2\""
Jul 29 16:02:38.893861 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.893835    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"]
Jul 29 16:02:38.938280 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.938244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbnm6\" (UniqueName: \"kubernetes.io/projected/22ff2446-ad31-4d16-8dc3-66caeb4279e9-kube-api-access-lbnm6\") pod \"payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:38.938463 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:38.938342    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff2446-ad31-4d16-8dc3-66caeb4279e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:39.039408 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.039360    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbnm6\" (UniqueName: \"kubernetes.io/projected/22ff2446-ad31-4d16-8dc3-66caeb4279e9-kube-api-access-lbnm6\") pod \"payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:39.039596 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.039505    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff2446-ad31-4d16-8dc3-66caeb4279e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:39.040157 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.040137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff2446-ad31-4d16-8dc3-66caeb4279e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:39.073393 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.073355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbnm6\" (UniqueName: \"kubernetes.io/projected/22ff2446-ad31-4d16-8dc3-66caeb4279e9-kube-api-access-lbnm6\") pod \"payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:39.195087 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.194974    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:39.332226 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.332169    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"]
Jul 29 16:02:39.334658 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:02:39.334631    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ff2446_ad31_4d16_8dc3_66caeb4279e9.slice/crio-112eec4f98d33a02f0962086b18a8b4c66295b01f34a041f0cb8cf2ba7072fde WatchSource:0}: Error finding container 112eec4f98d33a02f0962086b18a8b4c66295b01f34a041f0cb8cf2ba7072fde: Status 404 returned error can't find the container with id 112eec4f98d33a02f0962086b18a8b4c66295b01f34a041f0cb8cf2ba7072fde
Jul 29 16:02:39.455754 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.455650    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx" event={"ID":"22ff2446-ad31-4d16-8dc3-66caeb4279e9","Type":"ContainerStarted","Data":"18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225"}
Jul 29 16:02:39.455754 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.455719    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx" event={"ID":"22ff2446-ad31-4d16-8dc3-66caeb4279e9","Type":"ContainerStarted","Data":"112eec4f98d33a02f0962086b18a8b4c66295b01f34a041f0cb8cf2ba7072fde"}
Jul 29 16:02:39.455754 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.455750    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:02:39.473748 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:39.473694    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx" podStartSLOduration=1.473676289 podStartE2EDuration="1.473676289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:02:39.471414044 +0000 UTC m=+3097.765350866" watchObservedRunningTime="2026-07-29 16:02:39.473676289 +0000 UTC m=+3097.767613091"
Jul 29 16:02:43.581725 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:43.581687    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:02:50.462361 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:02:50.462329    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:03:04.322672 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.322634    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vv4mn"]
Jul 29 16:03:04.327384 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.327359    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:04.329917 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.329895    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hkl5k\""
Jul 29 16:03:04.334502 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.334472    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vv4mn"]
Jul 29 16:03:04.470518 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.470477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92wpd\" (UniqueName: \"kubernetes.io/projected/9b952934-f44d-444f-ab37-b4d510d8431e-kube-api-access-92wpd\") pod \"kuadrant-operator-catalog-vv4mn\" (UID: \"9b952934-f44d-444f-ab37-b4d510d8431e\") " pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:04.572024 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.571977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92wpd\" (UniqueName: \"kubernetes.io/projected/9b952934-f44d-444f-ab37-b4d510d8431e-kube-api-access-92wpd\") pod \"kuadrant-operator-catalog-vv4mn\" (UID: \"9b952934-f44d-444f-ab37-b4d510d8431e\") " pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:04.581913 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.581831    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92wpd\" (UniqueName: \"kubernetes.io/projected/9b952934-f44d-444f-ab37-b4d510d8431e-kube-api-access-92wpd\") pod \"kuadrant-operator-catalog-vv4mn\" (UID: \"9b952934-f44d-444f-ab37-b4d510d8431e\") " pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:04.638458 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.638417    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:04.775270 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:04.775246    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vv4mn"]
Jul 29 16:03:04.777197 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:03:04.777151    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b952934_f44d_444f_ab37_b4d510d8431e.slice/crio-f0c484bc05ec6b3567d300c5d703603f8e69d28924ba6680710bd38191fcbe23 WatchSource:0}: Error finding container f0c484bc05ec6b3567d300c5d703603f8e69d28924ba6680710bd38191fcbe23: Status 404 returned error can't find the container with id f0c484bc05ec6b3567d300c5d703603f8e69d28924ba6680710bd38191fcbe23
Jul 29 16:03:05.598077 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:05.598036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn" event={"ID":"9b952934-f44d-444f-ab37-b4d510d8431e","Type":"ContainerStarted","Data":"f0c484bc05ec6b3567d300c5d703603f8e69d28924ba6680710bd38191fcbe23"}
Jul 29 16:03:08.171736 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.171650    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"]
Jul 29 16:03:08.175442 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.175424    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.178027 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.177997    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-7e5daf-dockercfg-nnh2r\""
Jul 29 16:03:08.178151 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.178039    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-7e5daf\""
Jul 29 16:03:08.192349 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.192321    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"]
Jul 29 16:03:08.267170 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.267130    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"]
Jul 29 16:03:08.270749 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.270729    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.286225 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.286151    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"]
Jul 29 16:03:08.306139 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.306105    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngcz5\" (UniqueName: \"kubernetes.io/projected/db4379bb-c8b1-4b1b-a952-5ebf48165c86-kube-api-access-ngcz5\") pod \"payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.306328 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.306160    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4379bb-c8b1-4b1b-a952-5ebf48165c86-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.407625 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.407588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngcz5\" (UniqueName: \"kubernetes.io/projected/db4379bb-c8b1-4b1b-a952-5ebf48165c86-kube-api-access-ngcz5\") pod \"payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.407849 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.407640    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9wq4\" (UniqueName: \"kubernetes.io/projected/8e8e49da-75c2-4885-8cbc-ff32bea703b5-kube-api-access-h9wq4\") pod \"payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.407849 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.407670    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4379bb-c8b1-4b1b-a952-5ebf48165c86-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.407849 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.407771    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e8e49da-75c2-4885-8cbc-ff32bea703b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.408387 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.408370    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4379bb-c8b1-4b1b-a952-5ebf48165c86-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.417991 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.417963    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngcz5\" (UniqueName: \"kubernetes.io/projected/db4379bb-c8b1-4b1b-a952-5ebf48165c86-kube-api-access-ngcz5\") pod \"payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.486138 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.486036    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:08.508819 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.508772    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e8e49da-75c2-4885-8cbc-ff32bea703b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.508980 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.508922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9wq4\" (UniqueName: \"kubernetes.io/projected/8e8e49da-75c2-4885-8cbc-ff32bea703b5-kube-api-access-h9wq4\") pod \"payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.509625 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.509598    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e8e49da-75c2-4885-8cbc-ff32bea703b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.521109 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.521081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9wq4\" (UniqueName: \"kubernetes.io/projected/8e8e49da-75c2-4885-8cbc-ff32bea703b5-kube-api-access-h9wq4\") pod \"payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.581552 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.581500    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:08.625467 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.625429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn" event={"ID":"9b952934-f44d-444f-ab37-b4d510d8431e","Type":"ContainerStarted","Data":"0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56"}
Jul 29 16:03:08.631263 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.631233    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"]
Jul 29 16:03:08.633114 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:03:08.633087    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4379bb_c8b1_4b1b_a952_5ebf48165c86.slice/crio-802d6d303cb618ffc45f3f2827aed20741ba280c82c204a6251241b6e283ca46 WatchSource:0}: Error finding container 802d6d303cb618ffc45f3f2827aed20741ba280c82c204a6251241b6e283ca46: Status 404 returned error can't find the container with id 802d6d303cb618ffc45f3f2827aed20741ba280c82c204a6251241b6e283ca46
Jul 29 16:03:08.646074 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.646016    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn" podStartSLOduration=1.5359532150000001 podStartE2EDuration="4.645993529s" totalImagesPullingTime="3.110040314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:03:08.643738848 +0000 UTC m=+3126.937675663" watchObservedRunningTime="2026-07-29 16:03:08.645993529 +0000 UTC m=+3126.939930336"
Jul 29 16:03:08.731647 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:08.731615    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"]
Jul 29 16:03:08.733667 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:03:08.733639    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e8e49da_75c2_4885_8cbc_ff32bea703b5.slice/crio-c383f31b90f14ac3b482bf3e9e4f8ce5da88e0e9996f76d6f3ce701d0ca18de8 WatchSource:0}: Error finding container c383f31b90f14ac3b482bf3e9e4f8ce5da88e0e9996f76d6f3ce701d0ca18de8: Status 404 returned error can't find the container with id c383f31b90f14ac3b482bf3e9e4f8ce5da88e0e9996f76d6f3ce701d0ca18de8
Jul 29 16:03:09.632914 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.632874    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2" event={"ID":"8e8e49da-75c2-4885-8cbc-ff32bea703b5","Type":"ContainerStarted","Data":"81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c"}
Jul 29 16:03:09.633393 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.632932    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2" event={"ID":"8e8e49da-75c2-4885-8cbc-ff32bea703b5","Type":"ContainerStarted","Data":"c383f31b90f14ac3b482bf3e9e4f8ce5da88e0e9996f76d6f3ce701d0ca18de8"}
Jul 29 16:03:09.633393 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.633053    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:09.634486 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.634460    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c" event={"ID":"db4379bb-c8b1-4b1b-a952-5ebf48165c86","Type":"ContainerStarted","Data":"00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89"}
Jul 29 16:03:09.634614 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.634505    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c" event={"ID":"db4379bb-c8b1-4b1b-a952-5ebf48165c86","Type":"ContainerStarted","Data":"802d6d303cb618ffc45f3f2827aed20741ba280c82c204a6251241b6e283ca46"}
Jul 29 16:03:09.634614 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.634552    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:09.654150 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.654094    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2" podStartSLOduration=1.654080327 podStartE2EDuration="1.654080327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:03:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:03:09.651555393 +0000 UTC m=+3127.945492207" watchObservedRunningTime="2026-07-29 16:03:09.654080327 +0000 UTC m=+3127.948017129"
Jul 29 16:03:09.666994 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:09.666923    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c" podStartSLOduration=1.66690392 podStartE2EDuration="1.66690392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:03:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:03:09.666301642 +0000 UTC m=+3127.960238441" watchObservedRunningTime="2026-07-29 16:03:09.66690392 +0000 UTC m=+3127.960840722"
Jul 29 16:03:14.639233 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:14.639174    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:14.639745 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:14.639257    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:14.661951 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:14.661923    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:14.684677 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:14.684646    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:14.691608 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:14.691580    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:03:14.892389 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:14.892291    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vv4mn"]
Jul 29 16:03:16.674179 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:16.674132    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn" podUID="9b952934-f44d-444f-ab37-b4d510d8431e" containerName="registry-server" containerID="cri-o://0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56" gracePeriod=2
Jul 29 16:03:16.921383 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:16.921352    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:16.988357 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:16.988262    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92wpd\" (UniqueName: \"kubernetes.io/projected/9b952934-f44d-444f-ab37-b4d510d8431e-kube-api-access-92wpd\") pod \"9b952934-f44d-444f-ab37-b4d510d8431e\" (UID: \"9b952934-f44d-444f-ab37-b4d510d8431e\") "
Jul 29 16:03:16.990438 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:16.990404    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b952934-f44d-444f-ab37-b4d510d8431e-kube-api-access-92wpd" (OuterVolumeSpecName: "kube-api-access-92wpd") pod "9b952934-f44d-444f-ab37-b4d510d8431e" (UID: "9b952934-f44d-444f-ab37-b4d510d8431e"). InnerVolumeSpecName "kube-api-access-92wpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:03:17.089057 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.089014    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92wpd\" (UniqueName: \"kubernetes.io/projected/9b952934-f44d-444f-ab37-b4d510d8431e-kube-api-access-92wpd\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:03:17.680976 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.680939    2560 generic.go:356] "Generic (PLEG): container finished" podID="9b952934-f44d-444f-ab37-b4d510d8431e" containerID="0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56" exitCode=0
Jul 29 16:03:17.680976 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.680982    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn" event={"ID":"9b952934-f44d-444f-ab37-b4d510d8431e","Type":"ContainerDied","Data":"0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56"}
Jul 29 16:03:17.681573 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.680991    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn"
Jul 29 16:03:17.681573 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.681022    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vv4mn" event={"ID":"9b952934-f44d-444f-ab37-b4d510d8431e","Type":"ContainerDied","Data":"f0c484bc05ec6b3567d300c5d703603f8e69d28924ba6680710bd38191fcbe23"}
Jul 29 16:03:17.681573 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.681043    2560 scope.go:117] "RemoveContainer" containerID="0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56"
Jul 29 16:03:17.691095 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.691074    2560 scope.go:117] "RemoveContainer" containerID="0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56"
Jul 29 16:03:17.691452 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:03:17.691431    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56\": container with ID starting with 0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56 not found: ID does not exist" containerID="0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56"
Jul 29 16:03:17.691507 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.691461    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56"} err="failed to get container status \"0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56\": rpc error: code = NotFound desc = could not find container \"0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56\": container with ID starting with 0c55d7fc539a89ad616eeb5643c2bb4f6e9c8d901ea4003fba85c3b386fbcb56 not found: ID does not exist"
Jul 29 16:03:17.705216 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.705164    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vv4mn"]
Jul 29 16:03:17.710546 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:17.710513    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vv4mn"]
Jul 29 16:03:18.166756 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:18.166712    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b952934-f44d-444f-ab37-b4d510d8431e" path="/var/lib/kubelet/pods/9b952934-f44d-444f-ab37-b4d510d8431e/volumes"
Jul 29 16:03:20.642024 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:20.641991    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:03:20.642495 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:20.642146    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:03:46.684044 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.684000    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"]
Jul 29 16:03:46.684496 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.684406    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b952934-f44d-444f-ab37-b4d510d8431e" containerName="registry-server"
Jul 29 16:03:46.684496 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.684418    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b952934-f44d-444f-ab37-b4d510d8431e" containerName="registry-server"
Jul 29 16:03:46.684496 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.684485    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b952934-f44d-444f-ab37-b4d510d8431e" containerName="registry-server"
Jul 29 16:03:46.687698 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.687678    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.691170 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.691143    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"openshift-service-ca.crt\""
Jul 29 16:03:46.691327 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.691274    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"auth-test-model-02add2-kserve-self-signed-certs\""
Jul 29 16:03:46.691327 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.691306    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"kube-root-ca.crt\""
Jul 29 16:03:46.691327 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.691275    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"default-dockercfg-8q2k7\""
Jul 29 16:03:46.696935 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.696911    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"]
Jul 29 16:03:46.757312 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.757275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-home\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.757312 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.757317    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-tmp-dir\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.757544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.757350    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-model-cache\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.757544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.757408    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c97dbe5b-954f-4454-acfb-6c51d44295db-tls-certs\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.757544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.757438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-dshm\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.757544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.757455    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrnp8\" (UniqueName: \"kubernetes.io/projected/c97dbe5b-954f-4454-acfb-6c51d44295db-kube-api-access-vrnp8\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858057 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858024    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-home\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858057 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-tmp-dir\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858340 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858085    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-model-cache\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858340 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858117    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c97dbe5b-954f-4454-acfb-6c51d44295db-tls-certs\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858340 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858141    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-dshm\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858340 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrnp8\" (UniqueName: \"kubernetes.io/projected/c97dbe5b-954f-4454-acfb-6c51d44295db-kube-api-access-vrnp8\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858569 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-tmp-dir\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858569 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-model-cache\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.858673 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.858604    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-home\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.860467 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.860439    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-dshm\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.860616 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.860597    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c97dbe5b-954f-4454-acfb-6c51d44295db-tls-certs\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.868371 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.868344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrnp8\" (UniqueName: \"kubernetes.io/projected/c97dbe5b-954f-4454-acfb-6c51d44295db-kube-api-access-vrnp8\") pod \"auth-test-model-02add2-kserve-6cc4855599-t8f69\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") " pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:46.998132 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:46.998024    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:47.143653 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:47.143628    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"]
Jul 29 16:03:47.145432 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:03:47.145398    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc97dbe5b_954f_4454_acfb_6c51d44295db.slice/crio-fda13cbf7acca58df7b65de2dcf67449898a4a59d77c6f4e56971ec89e222b7e WatchSource:0}: Error finding container fda13cbf7acca58df7b65de2dcf67449898a4a59d77c6f4e56971ec89e222b7e: Status 404 returned error can't find the container with id fda13cbf7acca58df7b65de2dcf67449898a4a59d77c6f4e56971ec89e222b7e
Jul 29 16:03:47.865881 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:47.865846    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69" event={"ID":"c97dbe5b-954f-4454-acfb-6c51d44295db","Type":"ContainerStarted","Data":"383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25"}
Jul 29 16:03:47.865881 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:47.865888    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69" event={"ID":"c97dbe5b-954f-4454-acfb-6c51d44295db","Type":"ContainerStarted","Data":"fda13cbf7acca58df7b65de2dcf67449898a4a59d77c6f4e56971ec89e222b7e"}
Jul 29 16:03:47.886192 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:47.886126    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69" podStartSLOduration=1.886110019 podStartE2EDuration="1.886110019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:03:47.885283467 +0000 UTC m=+3166.179220281" watchObservedRunningTime="2026-07-29 16:03:47.886110019 +0000 UTC m=+3166.180046821"
Jul 29 16:03:50.081852 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:50.081808    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:03:56.998246 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:56.998201    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:56.998742 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:56.998264    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:57.011027 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:57.010997    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:03:57.937950 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:03:57.937914    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:04:17.209695 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.209652    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"]
Jul 29 16:04:17.213443 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.213417    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.216717 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.216690    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"kube-root-ca.crt\""
Jul 29 16:04:17.216871 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.216740    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"openshift-service-ca.crt\""
Jul 29 16:04:17.216871 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.216751    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"auth-test-model-7e5daf-kserve-self-signed-certs\""
Jul 29 16:04:17.216871 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.216752    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"default-dockercfg-bqcmn\""
Jul 29 16:04:17.224160 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.224135    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"]
Jul 29 16:04:17.317708 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.317662    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-dshm\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.317708 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.317710    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tls-certs\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.317934 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.317777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tmp-dir\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.317934 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.317816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-model-cache\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.317934 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.317869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-home\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.317934 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.317906    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hzfw\" (UniqueName: \"kubernetes.io/projected/5ecb8c89-53db-470b-8b84-9ae7f14c8294-kube-api-access-9hzfw\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419123 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419080    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tmp-dir\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419123 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419123    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-model-cache\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419432 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-home\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419432 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419215    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hzfw\" (UniqueName: \"kubernetes.io/projected/5ecb8c89-53db-470b-8b84-9ae7f14c8294-kube-api-access-9hzfw\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419432 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419253    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-dshm\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419432 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tls-certs\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419628 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419587    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tmp-dir\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419628 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-home\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.419708 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.419632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-model-cache\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.421681 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.421657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-dshm\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.421807 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.421790    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tls-certs\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.431790 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.431756    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hzfw\" (UniqueName: \"kubernetes.io/projected/5ecb8c89-53db-470b-8b84-9ae7f14c8294-kube-api-access-9hzfw\") pod \"auth-test-model-7e5daf-kserve-8fff47bc-rj8ct\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") " pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.523971 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.523874    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:17.661827 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:17.661791    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"]
Jul 29 16:04:17.663857 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:04:17.663832    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ecb8c89_53db_470b_8b84_9ae7f14c8294.slice/crio-e34b58574024ca92c36b03b1a66dde0dfd3b9dd33ca0655436f2443084a8e7c5 WatchSource:0}: Error finding container e34b58574024ca92c36b03b1a66dde0dfd3b9dd33ca0655436f2443084a8e7c5: Status 404 returned error can't find the container with id e34b58574024ca92c36b03b1a66dde0dfd3b9dd33ca0655436f2443084a8e7c5
Jul 29 16:04:18.036737 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:18.036695    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct" event={"ID":"5ecb8c89-53db-470b-8b84-9ae7f14c8294","Type":"ContainerStarted","Data":"b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe"}
Jul 29 16:04:18.036737 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:18.036743    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct" event={"ID":"5ecb8c89-53db-470b-8b84-9ae7f14c8294","Type":"ContainerStarted","Data":"e34b58574024ca92c36b03b1a66dde0dfd3b9dd33ca0655436f2443084a8e7c5"}
Jul 29 16:04:18.061270 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:18.061209    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct" podStartSLOduration=1.061192101 podStartE2EDuration="1.061192101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:04:18.058814396 +0000 UTC m=+3196.352751199" watchObservedRunningTime="2026-07-29 16:04:18.061192101 +0000 UTC m=+3196.355128894"
Jul 29 16:04:20.886277 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:20.886237    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:04:27.524882 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:27.524835    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:27.525464 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:27.524912    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:27.537622 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:27.537589    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:28.104076 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:28.104048    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:04:59.787309 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:04:59.787224    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:05:25.491978 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:05:25.491940    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:05:41.994446 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:05:41.994412    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:06:02.413161 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:06:02.413125    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:06:02.431502 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:06:02.431473    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:06:02.441411 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:06:02.441386    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:06:02.449364 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:06:02.449339    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:06:09.693839 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:06:09.693796    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:06:26.593416 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:06:26.593323    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:06:53.991933 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:06:53.991889    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:07:02.618309 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:07:02.618276    2560 scope.go:117] "RemoveContainer" containerID="c7e347ac185a73329414b1e8cfcb5ce71b6c9e08ee64a995c765c76ca9212e2b"
Jul 29 16:07:12.589783 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:07:12.589746    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:07:39.595753 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:07:39.595715    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:07:57.878851 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:07:57.878753    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:08:02.641059 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:08:02.641015    2560 scope.go:117] "RemoveContainer" containerID="5b5b92ed7ac86b0b6753cb85ad2153746f4e98740d0c3538c4c4a4dbd785dc9f"
Jul 29 16:08:23.083274 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:08:23.083235    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:08:37.091530 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:08:37.091482    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:08:49.582886 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:08:49.582850    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:09:10.582215 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:10.581691    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:09:14.753582 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:14.753536    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"]
Jul 29 16:09:15.692813 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:15.692777    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"]
Jul 29 16:09:27.488465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:27.488429    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:09:29.772923 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:29.772875    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69" podUID="c97dbe5b-954f-4454-acfb-6c51d44295db" containerName="main" containerID="cri-o://383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25" gracePeriod=45
Jul 29 16:09:30.020856 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.020825    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:09:30.055021 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.054943    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-dshm\") pod \"c97dbe5b-954f-4454-acfb-6c51d44295db\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") "
Jul 29 16:09:30.055021 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.054992    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-home\") pod \"c97dbe5b-954f-4454-acfb-6c51d44295db\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") "
Jul 29 16:09:30.055021 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055021    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-tmp-dir\") pod \"c97dbe5b-954f-4454-acfb-6c51d44295db\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") "
Jul 29 16:09:30.055326 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055058    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-model-cache\") pod \"c97dbe5b-954f-4454-acfb-6c51d44295db\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") "
Jul 29 16:09:30.055326 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055091    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c97dbe5b-954f-4454-acfb-6c51d44295db-tls-certs\") pod \"c97dbe5b-954f-4454-acfb-6c51d44295db\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") "
Jul 29 16:09:30.055326 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055136    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrnp8\" (UniqueName: \"kubernetes.io/projected/c97dbe5b-954f-4454-acfb-6c51d44295db-kube-api-access-vrnp8\") pod \"c97dbe5b-954f-4454-acfb-6c51d44295db\" (UID: \"c97dbe5b-954f-4454-acfb-6c51d44295db\") "
Jul 29 16:09:30.055491 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055329    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-home" (OuterVolumeSpecName: "home") pod "c97dbe5b-954f-4454-acfb-6c51d44295db" (UID: "c97dbe5b-954f-4454-acfb-6c51d44295db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:30.055491 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055342    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c97dbe5b-954f-4454-acfb-6c51d44295db" (UID: "c97dbe5b-954f-4454-acfb-6c51d44295db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:30.055491 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055422    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-model-cache" (OuterVolumeSpecName: "model-cache") pod "c97dbe5b-954f-4454-acfb-6c51d44295db" (UID: "c97dbe5b-954f-4454-acfb-6c51d44295db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:30.055630 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055523    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:30.055630 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055543    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:30.055630 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.055557    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:30.057366 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.057342    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-dshm" (OuterVolumeSpecName: "dshm") pod "c97dbe5b-954f-4454-acfb-6c51d44295db" (UID: "c97dbe5b-954f-4454-acfb-6c51d44295db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:30.057487 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.057392    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c97dbe5b-954f-4454-acfb-6c51d44295db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c97dbe5b-954f-4454-acfb-6c51d44295db" (UID: "c97dbe5b-954f-4454-acfb-6c51d44295db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:09:30.057689 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.057673    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c97dbe5b-954f-4454-acfb-6c51d44295db-kube-api-access-vrnp8" (OuterVolumeSpecName: "kube-api-access-vrnp8") pod "c97dbe5b-954f-4454-acfb-6c51d44295db" (UID: "c97dbe5b-954f-4454-acfb-6c51d44295db"). InnerVolumeSpecName "kube-api-access-vrnp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:09:30.156505 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.156465    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c97dbe5b-954f-4454-acfb-6c51d44295db-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:30.156505 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.156500    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrnp8\" (UniqueName: \"kubernetes.io/projected/c97dbe5b-954f-4454-acfb-6c51d44295db-kube-api-access-vrnp8\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:30.156505 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.156513    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c97dbe5b-954f-4454-acfb-6c51d44295db-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:30.712129 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.712061    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct" podUID="5ecb8c89-53db-470b-8b84-9ae7f14c8294" containerName="main" containerID="cri-o://b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe" gracePeriod=45
Jul 29 16:09:30.792357 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.792326    2560 generic.go:356] "Generic (PLEG): container finished" podID="c97dbe5b-954f-4454-acfb-6c51d44295db" containerID="383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25" exitCode=0
Jul 29 16:09:30.792764 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.792380    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"
Jul 29 16:09:30.792764 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.792410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69" event={"ID":"c97dbe5b-954f-4454-acfb-6c51d44295db","Type":"ContainerDied","Data":"383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25"}
Jul 29 16:09:30.792764 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.792454    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69" event={"ID":"c97dbe5b-954f-4454-acfb-6c51d44295db","Type":"ContainerDied","Data":"fda13cbf7acca58df7b65de2dcf67449898a4a59d77c6f4e56971ec89e222b7e"}
Jul 29 16:09:30.792764 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.792475    2560 scope.go:117] "RemoveContainer" containerID="383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25"
Jul 29 16:09:30.812580 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.812544    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"]
Jul 29 16:09:30.814783 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.814763    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/auth-test-model-02add2-kserve-6cc4855599-t8f69"]
Jul 29 16:09:30.824139 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.824111    2560 scope.go:117] "RemoveContainer" containerID="383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25"
Jul 29 16:09:30.824512 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:09:30.824486    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25\": container with ID starting with 383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25 not found: ID does not exist" containerID="383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25"
Jul 29 16:09:30.824625 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.824520    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25"} err="failed to get container status \"383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25\": rpc error: code = NotFound desc = could not find container \"383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25\": container with ID starting with 383b7d05b8104cb30a1c9f480522b8ff202c0cff6dddb39e02100318e85c2c25 not found: ID does not exist"
Jul 29 16:09:30.974790 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:30.974725    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:09:31.064750 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.064711    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tls-certs\") pod \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") "
Jul 29 16:09:31.064949 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.064761    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tmp-dir\") pod \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") "
Jul 29 16:09:31.064949 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.064797    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hzfw\" (UniqueName: \"kubernetes.io/projected/5ecb8c89-53db-470b-8b84-9ae7f14c8294-kube-api-access-9hzfw\") pod \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") "
Jul 29 16:09:31.064949 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.064869    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-dshm\") pod \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") "
Jul 29 16:09:31.064949 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.064892    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-model-cache\") pod \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") "
Jul 29 16:09:31.064949 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.064923    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-home\") pod \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\" (UID: \"5ecb8c89-53db-470b-8b84-9ae7f14c8294\") "
Jul 29 16:09:31.065296 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.065138    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ecb8c89-53db-470b-8b84-9ae7f14c8294" (UID: "5ecb8c89-53db-470b-8b84-9ae7f14c8294"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:31.065296 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.065262    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ecb8c89-53db-470b-8b84-9ae7f14c8294" (UID: "5ecb8c89-53db-470b-8b84-9ae7f14c8294"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:31.065296 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.065275    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-home" (OuterVolumeSpecName: "home") pod "5ecb8c89-53db-470b-8b84-9ae7f14c8294" (UID: "5ecb8c89-53db-470b-8b84-9ae7f14c8294"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:31.065296 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.065270    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:31.067211 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.067171    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-dshm" (OuterVolumeSpecName: "dshm") pod "5ecb8c89-53db-470b-8b84-9ae7f14c8294" (UID: "5ecb8c89-53db-470b-8b84-9ae7f14c8294"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:09:31.067335 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.067228    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ecb8c89-53db-470b-8b84-9ae7f14c8294-kube-api-access-9hzfw" (OuterVolumeSpecName: "kube-api-access-9hzfw") pod "5ecb8c89-53db-470b-8b84-9ae7f14c8294" (UID: "5ecb8c89-53db-470b-8b84-9ae7f14c8294"). InnerVolumeSpecName "kube-api-access-9hzfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:09:31.067335 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.067287    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ecb8c89-53db-470b-8b84-9ae7f14c8294" (UID: "5ecb8c89-53db-470b-8b84-9ae7f14c8294"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:09:31.166486 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.166444    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecb8c89-53db-470b-8b84-9ae7f14c8294-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:31.166486 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.166479    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hzfw\" (UniqueName: \"kubernetes.io/projected/5ecb8c89-53db-470b-8b84-9ae7f14c8294-kube-api-access-9hzfw\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:31.166486 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.166489    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:31.166730 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.166501    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:31.166730 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.166510    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8c89-53db-470b-8b84-9ae7f14c8294-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:09:31.798512 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.798469    2560 generic.go:356] "Generic (PLEG): container finished" podID="5ecb8c89-53db-470b-8b84-9ae7f14c8294" containerID="b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe" exitCode=0
Jul 29 16:09:31.798945 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.798520    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"
Jul 29 16:09:31.798945 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.798557    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct" event={"ID":"5ecb8c89-53db-470b-8b84-9ae7f14c8294","Type":"ContainerDied","Data":"b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe"}
Jul 29 16:09:31.798945 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.798604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct" event={"ID":"5ecb8c89-53db-470b-8b84-9ae7f14c8294","Type":"ContainerDied","Data":"e34b58574024ca92c36b03b1a66dde0dfd3b9dd33ca0655436f2443084a8e7c5"}
Jul 29 16:09:31.798945 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.798621    2560 scope.go:117] "RemoveContainer" containerID="b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe"
Jul 29 16:09:31.808401 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.808380    2560 scope.go:117] "RemoveContainer" containerID="b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe"
Jul 29 16:09:31.808663 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:09:31.808645    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe\": container with ID starting with b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe not found: ID does not exist" containerID="b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe"
Jul 29 16:09:31.808728 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.808674    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe"} err="failed to get container status \"b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe\": rpc error: code = NotFound desc = could not find container \"b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe\": container with ID starting with b979c10a1286398e1b0bcfbe18754d7c59846c1d56b8a99019864f83a4fb1bbe not found: ID does not exist"
Jul 29 16:09:31.821364 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.821333    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"]
Jul 29 16:09:31.826627 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:31.826601    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/auth-test-model-7e5daf-kserve-8fff47bc-rj8ct"]
Jul 29 16:09:32.166610 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:32.166580    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ecb8c89-53db-470b-8b84-9ae7f14c8294" path="/var/lib/kubelet/pods/5ecb8c89-53db-470b-8b84-9ae7f14c8294/volumes"
Jul 29 16:09:32.166996 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:32.166983    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c97dbe5b-954f-4454-acfb-6c51d44295db" path="/var/lib/kubelet/pods/c97dbe5b-954f-4454-acfb-6c51d44295db/volumes"
Jul 29 16:09:40.802120 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.802082    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"]
Jul 29 16:09:40.802620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.802465    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c97dbe5b-954f-4454-acfb-6c51d44295db" containerName="main"
Jul 29 16:09:40.802620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.802478    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c97dbe5b-954f-4454-acfb-6c51d44295db" containerName="main"
Jul 29 16:09:40.802620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.802506    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ecb8c89-53db-470b-8b84-9ae7f14c8294" containerName="main"
Jul 29 16:09:40.802620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.802511    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ecb8c89-53db-470b-8b84-9ae7f14c8294" containerName="main"
Jul 29 16:09:40.802620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.802566    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ecb8c89-53db-470b-8b84-9ae7f14c8294" containerName="main"
Jul 29 16:09:40.802620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.802577    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c97dbe5b-954f-4454-acfb-6c51d44295db" containerName="main"
Jul 29 16:09:40.807036 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.807007    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.812199 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.811089    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"openshift-service-ca.crt\""
Jul 29 16:09:40.812199 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.811421    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"kube-root-ca.crt\""
Jul 29 16:09:40.812199 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.811709    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"sub-test-model-02add2-kserve-self-signed-certs\""
Jul 29 16:09:40.812199 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.811937    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"default-dockercfg-8q2k7\""
Jul 29 16:09:40.819065 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.819036    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"]
Jul 29 16:09:40.855122 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.855088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-dshm\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.855361 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.855165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-model-cache\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.855361 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.855234    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tmp-dir\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.855361 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.855262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2tst\" (UniqueName: \"kubernetes.io/projected/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-kube-api-access-k2tst\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.855361 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.855293    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-home\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.855361 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.855319    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tls-certs\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956360 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956323    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tmp-dir\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956360 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956367    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2tst\" (UniqueName: \"kubernetes.io/projected/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-kube-api-access-k2tst\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956404    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-home\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956446    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tls-certs\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956484    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-dshm\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-model-cache\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956843 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tmp-dir\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956889 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956867    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-home\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.956951 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.956887    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-model-cache\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.958814 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.958784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-dshm\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.959118 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.959098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tls-certs\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:40.967821 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:40.967794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2tst\" (UniqueName: \"kubernetes.io/projected/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-kube-api-access-k2tst\") pod \"sub-test-model-02add2-kserve-b4b6dd45c-4prdc\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") " pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:41.122799 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:41.122764    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:41.266511 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:41.266484    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"]
Jul 29 16:09:41.268438 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:09:41.268406    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f2fcf67_103a_4ce9_9f7e_0483c82fd0ca.slice/crio-d7440eb0d5d6ddc55261f53c7d4743f498b9e42018e0d882c03a3ad06b4a1f98 WatchSource:0}: Error finding container d7440eb0d5d6ddc55261f53c7d4743f498b9e42018e0d882c03a3ad06b4a1f98: Status 404 returned error can't find the container with id d7440eb0d5d6ddc55261f53c7d4743f498b9e42018e0d882c03a3ad06b4a1f98
Jul 29 16:09:41.865625 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:41.865587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc" event={"ID":"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca","Type":"ContainerStarted","Data":"42fdc796fe4c817e664dc5315a3d35b362c731416204076ca9c448879fac2bd2"}
Jul 29 16:09:41.865625 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:41.865630    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc" event={"ID":"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca","Type":"ContainerStarted","Data":"d7440eb0d5d6ddc55261f53c7d4743f498b9e42018e0d882c03a3ad06b4a1f98"}
Jul 29 16:09:41.884520 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:41.884446    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc" podStartSLOduration=1.884426807 podStartE2EDuration="1.884426807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:09:41.883806639 +0000 UTC m=+3520.177743439" watchObservedRunningTime="2026-07-29 16:09:41.884426807 +0000 UTC m=+3520.178363663"
Jul 29 16:09:44.093226 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:44.093174    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:09:51.123062 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:51.123015    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:51.123062 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:51.123070    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:51.135878 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:51.135852    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:09:51.930443 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:09:51.930414    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:10:04.514684 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.514643    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"]
Jul 29 16:10:04.519544 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.519522    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.522729 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.522688    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"openshift-service-ca.crt\""
Jul 29 16:10:04.522919 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.522727    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"default-dockercfg-bqcmn\""
Jul 29 16:10:04.522919 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.522730    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"kube-root-ca.crt\""
Jul 29 16:10:04.522919 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.522782    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"sub-test-model-7e5daf-kserve-self-signed-certs\""
Jul 29 16:10:04.530930 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.530903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"]
Jul 29 16:10:04.674038 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.673996    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-dshm\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.674038 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.674039    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tls-certs\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.674320 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.674059    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-home\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.674320 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.674178    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-model-cache\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.674320 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.674218    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lvf\" (UniqueName: \"kubernetes.io/projected/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-kube-api-access-84lvf\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.674320 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.674249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tmp-dir\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.775601 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.775505    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-dshm\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.775601 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.775548    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tls-certs\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.775601 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.775567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-home\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.775867 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.775649    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-model-cache\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.775867 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.775680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84lvf\" (UniqueName: \"kubernetes.io/projected/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-kube-api-access-84lvf\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.775867 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.775702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tmp-dir\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.776080 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.776050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-home\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.776252 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.776088    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-model-cache\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.776252 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.776127    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tmp-dir\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.777845 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.777826    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-dshm\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.778296 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.778276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tls-certs\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.787161 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.787136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lvf\" (UniqueName: \"kubernetes.io/projected/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-kube-api-access-84lvf\") pod \"sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") " pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.831070 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.831035    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:04.974766 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.974740    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"]
Jul 29 16:10:04.976350 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:10:04.976318    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod430e3b40_f6b4_49ff_861c_d5b5f6ac9ed1.slice/crio-97de9e15d2aa8bcb618f319dcb714014929b534fc220324e9f294baa5ee81510 WatchSource:0}: Error finding container 97de9e15d2aa8bcb618f319dcb714014929b534fc220324e9f294baa5ee81510: Status 404 returned error can't find the container with id 97de9e15d2aa8bcb618f319dcb714014929b534fc220324e9f294baa5ee81510
Jul 29 16:10:04.999583 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:04.999553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58" event={"ID":"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1","Type":"ContainerStarted","Data":"97de9e15d2aa8bcb618f319dcb714014929b534fc220324e9f294baa5ee81510"}
Jul 29 16:10:06.006250 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:06.006212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58" event={"ID":"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1","Type":"ContainerStarted","Data":"fed7218798f74a3e5bbb2a88d8a56d2ac1579f7ab20872e396f6d7526c2dd2fd"}
Jul 29 16:10:06.025665 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:06.025612    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58" podStartSLOduration=2.025596247 podStartE2EDuration="2.025596247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:10:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:10:06.02355489 +0000 UTC m=+3544.317491693" watchObservedRunningTime="2026-07-29 16:10:06.025596247 +0000 UTC m=+3544.319533106"
Jul 29 16:10:08.384200 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:08.384143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:10:14.831521 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:14.831471    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:14.831521 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:14.831528    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:14.844368 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:14.844340    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:10:15.067143 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:10:15.067104    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:11:02.457638 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:02.457609    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:11:02.465662 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:02.465633    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:11:02.477235 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:02.477211    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:11:02.484606 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:02.484580    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:11:05.189007 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:05.188966    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:11:25.083930 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:25.083890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:11:46.682504 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:46.682464    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:11:52.450024 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:52.449977    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"]
Jul 29 16:11:53.409151 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:11:53.409107    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"]
Jul 29 16:12:04.483565 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:04.483527    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:12:07.469289 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.469240    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc" podUID="6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" containerName="main" containerID="cri-o://42fdc796fe4c817e664dc5315a3d35b362c731416204076ca9c448879fac2bd2" gracePeriod=45
Jul 29 16:12:07.669289 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.669250    2560 generic.go:356] "Generic (PLEG): container finished" podID="6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" containerID="42fdc796fe4c817e664dc5315a3d35b362c731416204076ca9c448879fac2bd2" exitCode=0
Jul 29 16:12:07.669475 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.669314    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc" event={"ID":"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca","Type":"ContainerDied","Data":"42fdc796fe4c817e664dc5315a3d35b362c731416204076ca9c448879fac2bd2"}
Jul 29 16:12:07.727639 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.727578    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:12:07.791803 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.791764    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-home\") pod \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") "
Jul 29 16:12:07.791803 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.791802    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tls-certs\") pod \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") "
Jul 29 16:12:07.792053 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.791833    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tmp-dir\") pod \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") "
Jul 29 16:12:07.792053 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.791908    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-model-cache\") pod \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") "
Jul 29 16:12:07.792053 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.791941    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-dshm\") pod \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") "
Jul 29 16:12:07.792053 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.791974    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2tst\" (UniqueName: \"kubernetes.io/projected/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-kube-api-access-k2tst\") pod \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\" (UID: \"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca\") "
Jul 29 16:12:07.792318 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.792080    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-home" (OuterVolumeSpecName: "home") pod "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" (UID: "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:07.792318 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.792177    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" (UID: "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:07.792318 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.792223    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" (UID: "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:07.792454 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.792349    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:07.792454 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.792368    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:07.792454 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.792380    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:07.794075 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.794055    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" (UID: "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:12:07.794151 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.794070    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-dshm" (OuterVolumeSpecName: "dshm") pod "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" (UID: "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:07.794228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.794204    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-kube-api-access-k2tst" (OuterVolumeSpecName: "kube-api-access-k2tst") pod "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" (UID: "6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca"). InnerVolumeSpecName "kube-api-access-k2tst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:12:07.893500 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.893455    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:07.893500 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.893493    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2tst\" (UniqueName: \"kubernetes.io/projected/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-kube-api-access-k2tst\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:07.893500 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:07.893503    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:08.428835 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.428737    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58" podUID="430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" containerName="main" containerID="cri-o://fed7218798f74a3e5bbb2a88d8a56d2ac1579f7ab20872e396f6d7526c2dd2fd" gracePeriod=45
Jul 29 16:12:08.677507 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.677461    2560 generic.go:356] "Generic (PLEG): container finished" podID="430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" containerID="fed7218798f74a3e5bbb2a88d8a56d2ac1579f7ab20872e396f6d7526c2dd2fd" exitCode=0
Jul 29 16:12:08.677894 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.677543    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58" event={"ID":"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1","Type":"ContainerDied","Data":"fed7218798f74a3e5bbb2a88d8a56d2ac1579f7ab20872e396f6d7526c2dd2fd"}
Jul 29 16:12:08.677894 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.677596    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58" event={"ID":"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1","Type":"ContainerDied","Data":"97de9e15d2aa8bcb618f319dcb714014929b534fc220324e9f294baa5ee81510"}
Jul 29 16:12:08.677894 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.677608    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97de9e15d2aa8bcb618f319dcb714014929b534fc220324e9f294baa5ee81510"
Jul 29 16:12:08.679242 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.679152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc" event={"ID":"6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca","Type":"ContainerDied","Data":"d7440eb0d5d6ddc55261f53c7d4743f498b9e42018e0d882c03a3ad06b4a1f98"}
Jul 29 16:12:08.679242 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.679230    2560 scope.go:117] "RemoveContainer" containerID="42fdc796fe4c817e664dc5315a3d35b362c731416204076ca9c448879fac2bd2"
Jul 29 16:12:08.679242 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.679236    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"
Jul 29 16:12:08.695407 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.695383    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:12:08.706135 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.706104    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"]
Jul 29 16:12:08.706706 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.706688    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" containerName="main"
Jul 29 16:12:08.706801 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.706709    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" containerName="main"
Jul 29 16:12:08.706801 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.706734    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" containerName="main"
Jul 29 16:12:08.706801 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.706744    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" containerName="main"
Jul 29 16:12:08.706931 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.706849    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" containerName="main"
Jul 29 16:12:08.706931 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.706864    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" containerName="main"
Jul 29 16:12:08.710492 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.710469    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"]
Jul 29 16:12:08.710614 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.710553    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.713370 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.713339    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"rate-test-model-02add2-kserve-self-signed-certs\""
Jul 29 16:12:08.713489 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.713388    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"default-dockercfg-8q2k7\""
Jul 29 16:12:08.713745 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.713728    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"openshift-service-ca.crt\""
Jul 29 16:12:08.714277 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.714264    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-02add2\"/\"kube-root-ca.crt\""
Jul 29 16:12:08.719594 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.719572    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"]
Jul 29 16:12:08.722020 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.721994    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/sub-test-model-02add2-kserve-b4b6dd45c-4prdc"]
Jul 29 16:12:08.800697 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.800663    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-model-cache\") pod \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") "
Jul 29 16:12:08.800885 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.800709    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84lvf\" (UniqueName: \"kubernetes.io/projected/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-kube-api-access-84lvf\") pod \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") "
Jul 29 16:12:08.800885 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.800739    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tmp-dir\") pod \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") "
Jul 29 16:12:08.800885 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.800817    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tls-certs\") pod \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") "
Jul 29 16:12:08.800885 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.800848    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-dshm\") pod \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") "
Jul 29 16:12:08.801162 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.800922    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-home\") pod \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\" (UID: \"430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1\") "
Jul 29 16:12:08.801162 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801004    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-model-cache" (OuterVolumeSpecName: "model-cache") pod "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" (UID: "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:08.801162 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801063    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" (UID: "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:08.801162 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801085    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-dshm\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.801162 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801154    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-model-cache\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.801442 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801211    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-tmp-dir\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.801442 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-home\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.801442 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801373    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-home" (OuterVolumeSpecName: "home") pod "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" (UID: "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:08.801442 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801409    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24846591-f8fe-4edb-9e14-9601c0fc902a-tls-certs\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.801564 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801446    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l22v\" (UniqueName: \"kubernetes.io/projected/24846591-f8fe-4edb-9e14-9601c0fc902a-kube-api-access-6l22v\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.801564 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801541    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:08.801564 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801555    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:08.801662 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.801569    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:08.803118 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.803094    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-dshm" (OuterVolumeSpecName: "dshm") pod "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" (UID: "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:12:08.803268 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.803246    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-kube-api-access-84lvf" (OuterVolumeSpecName: "kube-api-access-84lvf") pod "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" (UID: "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1"). InnerVolumeSpecName "kube-api-access-84lvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:12:08.803387 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.803344    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" (UID: "430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:12:08.901966 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.901926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24846591-f8fe-4edb-9e14-9601c0fc902a-tls-certs\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.901966 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.901965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l22v\" (UniqueName: \"kubernetes.io/projected/24846591-f8fe-4edb-9e14-9601c0fc902a-kube-api-access-6l22v\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.902228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902001    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-dshm\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.902228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-model-cache\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.902228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902069    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-tmp-dir\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.902228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-home\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.902228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902171    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:08.902228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902204    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:08.902228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902218    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84lvf\" (UniqueName: \"kubernetes.io/projected/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1-kube-api-access-84lvf\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:12:08.902596 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-tmp-dir\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.902596 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902547    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-model-cache\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.902596 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.902577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-home\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.904351 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.904331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-dshm\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.904591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.904571    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24846591-f8fe-4edb-9e14-9601c0fc902a-tls-certs\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:08.912938 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:08.912914    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l22v\" (UniqueName: \"kubernetes.io/projected/24846591-f8fe-4edb-9e14-9601c0fc902a-kube-api-access-6l22v\") pod \"rate-test-model-02add2-kserve-5c6695c6bc-gb6qw\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") " pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:09.021758 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.021655    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:09.160581 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.160555    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"]
Jul 29 16:12:09.162825 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:12:09.162796    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24846591_f8fe_4edb_9e14_9601c0fc902a.slice/crio-21a6c283f8b42829ac89bc089927a6e562a15db6228b25043160d6b85ca776f2 WatchSource:0}: Error finding container 21a6c283f8b42829ac89bc089927a6e562a15db6228b25043160d6b85ca776f2: Status 404 returned error can't find the container with id 21a6c283f8b42829ac89bc089927a6e562a15db6228b25043160d6b85ca776f2
Jul 29 16:12:09.686966 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.686928    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw" event={"ID":"24846591-f8fe-4edb-9e14-9601c0fc902a","Type":"ContainerStarted","Data":"ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4"}
Jul 29 16:12:09.686966 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.686977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw" event={"ID":"24846591-f8fe-4edb-9e14-9601c0fc902a","Type":"ContainerStarted","Data":"21a6c283f8b42829ac89bc089927a6e562a15db6228b25043160d6b85ca776f2"}
Jul 29 16:12:09.687927 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.687908    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"
Jul 29 16:12:09.711200 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.711130    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw" podStartSLOduration=1.7111138700000001 podStartE2EDuration="1.71111387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:12:09.708586201 +0000 UTC m=+3668.002523044" watchObservedRunningTime="2026-07-29 16:12:09.71111387 +0000 UTC m=+3668.005050671"
Jul 29 16:12:09.723601 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.723570    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"]
Jul 29 16:12:09.728835 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:09.728808    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/sub-test-model-7e5daf-kserve-cb9f48cbc-mmq58"]
Jul 29 16:12:10.167989 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:10.167951    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1" path="/var/lib/kubelet/pods/430e3b40-f6b4-49ff-861c-d5b5f6ac9ed1/volumes"
Jul 29 16:12:10.168382 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:10.168367    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca" path="/var/lib/kubelet/pods/6f2fcf67-103a-4ce9-9f7e-0483c82fd0ca/volumes"
Jul 29 16:12:13.992377 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:13.992326    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:12:19.022060 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:19.022017    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:19.022060 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:19.022075    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:19.035358 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:19.035324    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:19.752608 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:19.752578    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:12:39.986837 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:39.986755    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"]
Jul 29 16:12:39.990407 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:39.990390    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:39.993858 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:39.993827    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"openshift-service-ca.crt\""
Jul 29 16:12:39.994002 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:39.993929    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"kube-root-ca.crt\""
Jul 29 16:12:39.994002 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:39.993981    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"rate-test-model-7e5daf-kserve-self-signed-certs\""
Jul 29 16:12:39.994136 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:39.994001    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7e5daf\"/\"default-dockercfg-bqcmn\""
Jul 29 16:12:40.000716 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.000693    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"]
Jul 29 16:12:40.090313 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.090277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77e6779-334d-4d78-a133-74fe54a08cca-tls-certs\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.090500 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.090340    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-home\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.090500 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.090389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-tmp-dir\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.090500 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.090481    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-model-cache\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.090613 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.090511    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-dshm\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.090613 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.090565    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/c77e6779-334d-4d78-a133-74fe54a08cca-kube-api-access-l47qz\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.191223 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.191157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/c77e6779-334d-4d78-a133-74fe54a08cca-kube-api-access-l47qz\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.191459 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.191273    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77e6779-334d-4d78-a133-74fe54a08cca-tls-certs\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.191548 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.191466    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-home\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.191837 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.191813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-home\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.191911 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.191825    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-tmp-dir\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.191911 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.191900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-model-cache\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.192005 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.191950    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-dshm\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.192090 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.192071    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-tmp-dir\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.192285 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.192266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-model-cache\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.194076 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.194056    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77e6779-334d-4d78-a133-74fe54a08cca-tls-certs\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.194172 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.194144    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-dshm\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.201346 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.201323    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/c77e6779-334d-4d78-a133-74fe54a08cca-kube-api-access-l47qz\") pod \"rate-test-model-7e5daf-kserve-545cc768b7-zz9jp\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") " pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.300912 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.300808    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:40.444569 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.444533    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"]
Jul 29 16:12:40.445575 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:12:40.445542    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc77e6779_334d_4d78_a133_74fe54a08cca.slice/crio-2d6c46f7ca9c7454fcc9a92ea7a7471099ef4a433419a6e023610d10bae7424b WatchSource:0}: Error finding container 2d6c46f7ca9c7454fcc9a92ea7a7471099ef4a433419a6e023610d10bae7424b: Status 404 returned error can't find the container with id 2d6c46f7ca9c7454fcc9a92ea7a7471099ef4a433419a6e023610d10bae7424b
Jul 29 16:12:40.862014 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.861974    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp" event={"ID":"c77e6779-334d-4d78-a133-74fe54a08cca","Type":"ContainerStarted","Data":"5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84"}
Jul 29 16:12:40.862254 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.862025    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp" event={"ID":"c77e6779-334d-4d78-a133-74fe54a08cca","Type":"ContainerStarted","Data":"2d6c46f7ca9c7454fcc9a92ea7a7471099ef4a433419a6e023610d10bae7424b"}
Jul 29 16:12:40.882631 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:40.882572    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp" podStartSLOduration=1.882553359 podStartE2EDuration="1.882553359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:12:40.88022014 +0000 UTC m=+3699.174156940" watchObservedRunningTime="2026-07-29 16:12:40.882553359 +0000 UTC m=+3699.176490160"
Jul 29 16:12:43.909910 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:43.909872    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:12:50.301676 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:50.301633    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:50.302227 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:50.301692    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:50.314248 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:50.314219    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:12:50.931399 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:12:50.931369    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:13:23.279268 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:13:23.279227    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:13:41.487172 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:13:41.487135    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:14:00.678935 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:00.678851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:14:06.329235 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:06.329198    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"]
Jul 29 16:14:07.395568 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:07.395534    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"]
Jul 29 16:14:08.465138 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.465101    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7831-140710491541040"]
Jul 29 16:14:08.468689 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.468671    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491541040"
Jul 29 16:14:08.471145 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.471125    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-97rc9\""
Jul 29 16:14:08.479151 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.479126    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710491541040"]
Jul 29 16:14:08.515000 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.514965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl6fp\" (UniqueName: \"kubernetes.io/projected/ed0df4db-1bbd-4629-abd0-f6aaa5e027e1-kube-api-access-bl6fp\") pod \"test-curl-7831-140710491541040\" (UID: \"ed0df4db-1bbd-4629-abd0-f6aaa5e027e1\") " pod="opendatahub/test-curl-7831-140710491541040"
Jul 29 16:14:08.616445 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.616402    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl6fp\" (UniqueName: \"kubernetes.io/projected/ed0df4db-1bbd-4629-abd0-f6aaa5e027e1-kube-api-access-bl6fp\") pod \"test-curl-7831-140710491541040\" (UID: \"ed0df4db-1bbd-4629-abd0-f6aaa5e027e1\") " pod="opendatahub/test-curl-7831-140710491541040"
Jul 29 16:14:08.626517 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.626486    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl6fp\" (UniqueName: \"kubernetes.io/projected/ed0df4db-1bbd-4629-abd0-f6aaa5e027e1-kube-api-access-bl6fp\") pod \"test-curl-7831-140710491541040\" (UID: \"ed0df4db-1bbd-4629-abd0-f6aaa5e027e1\") " pod="opendatahub/test-curl-7831-140710491541040"
Jul 29 16:14:08.779045 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.778941    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491541040"
Jul 29 16:14:08.920232 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.920199    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710491541040"]
Jul 29 16:14:08.922051 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:14:08.922017    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded0df4db_1bbd_4629_abd0_f6aaa5e027e1.slice/crio-bfdd39af1eb324307e90950d0b2be73bacfe3d5ca9e6872b6770a130e19f2e3b WatchSource:0}: Error finding container bfdd39af1eb324307e90950d0b2be73bacfe3d5ca9e6872b6770a130e19f2e3b: Status 404 returned error can't find the container with id bfdd39af1eb324307e90950d0b2be73bacfe3d5ca9e6872b6770a130e19f2e3b
Jul 29 16:14:08.923458 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:08.923440    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:14:09.352774 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:09.352737    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710491541040" event={"ID":"ed0df4db-1bbd-4629-abd0-f6aaa5e027e1","Type":"ContainerStarted","Data":"bfdd39af1eb324307e90950d0b2be73bacfe3d5ca9e6872b6770a130e19f2e3b"}
Jul 29 16:14:10.360667 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:10.360631    2560 generic.go:356] "Generic (PLEG): container finished" podID="ed0df4db-1bbd-4629-abd0-f6aaa5e027e1" containerID="fac82191259b674264fbd18cbc5497184731d1bbfdcfcf388b94ccc0bc9a6f27" exitCode=0
Jul 29 16:14:10.361118 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:10.360686    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710491541040" event={"ID":"ed0df4db-1bbd-4629-abd0-f6aaa5e027e1","Type":"ContainerDied","Data":"fac82191259b674264fbd18cbc5497184731d1bbfdcfcf388b94ccc0bc9a6f27"}
Jul 29 16:14:11.504019 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.503993    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491541040"
Jul 29 16:14:11.542507 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.542477    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7831-140710491541040_ed0df4db-1bbd-4629-abd0-f6aaa5e027e1/test-curl-7831-140710491541040/0.log"
Jul 29 16:14:11.545687 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.545667    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl6fp\" (UniqueName: \"kubernetes.io/projected/ed0df4db-1bbd-4629-abd0-f6aaa5e027e1-kube-api-access-bl6fp\") pod \"ed0df4db-1bbd-4629-abd0-f6aaa5e027e1\" (UID: \"ed0df4db-1bbd-4629-abd0-f6aaa5e027e1\") "
Jul 29 16:14:11.547856 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.547834    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed0df4db-1bbd-4629-abd0-f6aaa5e027e1-kube-api-access-bl6fp" (OuterVolumeSpecName: "kube-api-access-bl6fp") pod "ed0df4db-1bbd-4629-abd0-f6aaa5e027e1" (UID: "ed0df4db-1bbd-4629-abd0-f6aaa5e027e1"). InnerVolumeSpecName "kube-api-access-bl6fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:11.574690 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.574662    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7831-140710491541040"]
Jul 29 16:14:11.579275 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.579251    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7831-140710491541040"]
Jul 29 16:14:11.646805 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.646719    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl6fp\" (UniqueName: \"kubernetes.io/projected/ed0df4db-1bbd-4629-abd0-f6aaa5e027e1-kube-api-access-bl6fp\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:11.707399 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.707363    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7831-140710491538864"]
Jul 29 16:14:11.707768 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.707755    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0df4db-1bbd-4629-abd0-f6aaa5e027e1" containerName="test-curl-7831-140710491541040"
Jul 29 16:14:11.707816 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.707770    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0df4db-1bbd-4629-abd0-f6aaa5e027e1" containerName="test-curl-7831-140710491541040"
Jul 29 16:14:11.707863 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.707854    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed0df4db-1bbd-4629-abd0-f6aaa5e027e1" containerName="test-curl-7831-140710491541040"
Jul 29 16:14:11.710887 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.710868    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:11.716061 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.716036    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710491538864"]
Jul 29 16:14:11.748013 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.747985    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrvts\" (UniqueName: \"kubernetes.io/projected/c23c5063-0dce-402d-adbe-f4a95dc952bf-kube-api-access-lrvts\") pod \"test-curl-7831-140710491538864\" (UID: \"c23c5063-0dce-402d-adbe-f4a95dc952bf\") " pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:11.849369 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.849328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrvts\" (UniqueName: \"kubernetes.io/projected/c23c5063-0dce-402d-adbe-f4a95dc952bf-kube-api-access-lrvts\") pod \"test-curl-7831-140710491538864\" (UID: \"c23c5063-0dce-402d-adbe-f4a95dc952bf\") " pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:11.859072 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:11.859045    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrvts\" (UniqueName: \"kubernetes.io/projected/c23c5063-0dce-402d-adbe-f4a95dc952bf-kube-api-access-lrvts\") pod \"test-curl-7831-140710491538864\" (UID: \"c23c5063-0dce-402d-adbe-f4a95dc952bf\") " pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:12.021384 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:12.021284    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:12.157251 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:12.157219    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710491538864"]
Jul 29 16:14:12.158604 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:14:12.158578    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc23c5063_0dce_402d_adbe_f4a95dc952bf.slice/crio-3dd6b872f31a34ab52395e6406bbdc78591543b066247d93c68b9352712f7134 WatchSource:0}: Error finding container 3dd6b872f31a34ab52395e6406bbdc78591543b066247d93c68b9352712f7134: Status 404 returned error can't find the container with id 3dd6b872f31a34ab52395e6406bbdc78591543b066247d93c68b9352712f7134
Jul 29 16:14:12.167707 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:12.167677    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed0df4db-1bbd-4629-abd0-f6aaa5e027e1" path="/var/lib/kubelet/pods/ed0df4db-1bbd-4629-abd0-f6aaa5e027e1/volumes"
Jul 29 16:14:12.374032 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:12.374002    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491541040"
Jul 29 16:14:12.374257 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:12.374059    2560 scope.go:117] "RemoveContainer" containerID="fac82191259b674264fbd18cbc5497184731d1bbfdcfcf388b94ccc0bc9a6f27"
Jul 29 16:14:12.375431 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:12.375313    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710491538864" event={"ID":"c23c5063-0dce-402d-adbe-f4a95dc952bf","Type":"ContainerStarted","Data":"3dd6b872f31a34ab52395e6406bbdc78591543b066247d93c68b9352712f7134"}
Jul 29 16:14:13.382902 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:13.382868    2560 generic.go:356] "Generic (PLEG): container finished" podID="c23c5063-0dce-402d-adbe-f4a95dc952bf" containerID="823f4a349702dae132ebb502ce98fc31f01afde49bf1799cf39f27d5ad695da8" exitCode=0
Jul 29 16:14:13.383367 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:13.382951    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710491538864" event={"ID":"c23c5063-0dce-402d-adbe-f4a95dc952bf","Type":"ContainerDied","Data":"823f4a349702dae132ebb502ce98fc31f01afde49bf1799cf39f27d5ad695da8"}
Jul 29 16:14:13.892034 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:13.891992    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:14:14.531170 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.531143    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:14.553632 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.553603    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7831-140710491538864_c23c5063-0dce-402d-adbe-f4a95dc952bf/test-curl-7831-140710491538864/0.log"
Jul 29 16:14:14.576438 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.576403    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrvts\" (UniqueName: \"kubernetes.io/projected/c23c5063-0dce-402d-adbe-f4a95dc952bf-kube-api-access-lrvts\") pod \"c23c5063-0dce-402d-adbe-f4a95dc952bf\" (UID: \"c23c5063-0dce-402d-adbe-f4a95dc952bf\") "
Jul 29 16:14:14.578509 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.578477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23c5063-0dce-402d-adbe-f4a95dc952bf-kube-api-access-lrvts" (OuterVolumeSpecName: "kube-api-access-lrvts") pod "c23c5063-0dce-402d-adbe-f4a95dc952bf" (UID: "c23c5063-0dce-402d-adbe-f4a95dc952bf"). InnerVolumeSpecName "kube-api-access-lrvts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:14.583646 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.583619    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7831-140710491538864"]
Jul 29 16:14:14.587578 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.587558    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7831-140710491538864"]
Jul 29 16:14:14.677446 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.677355    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrvts\" (UniqueName: \"kubernetes.io/projected/c23c5063-0dce-402d-adbe-f4a95dc952bf-kube-api-access-lrvts\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:14.755875 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.755838    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:14.756248 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.756233    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23c5063-0dce-402d-adbe-f4a95dc952bf" containerName="test-curl-7831-140710491538864"
Jul 29 16:14:14.756313 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.756249    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23c5063-0dce-402d-adbe-f4a95dc952bf" containerName="test-curl-7831-140710491538864"
Jul 29 16:14:14.756351 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.756323    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23c5063-0dce-402d-adbe-f4a95dc952bf" containerName="test-curl-7831-140710491538864"
Jul 29 16:14:14.759256 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.759238    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:14.769458 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.769435    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:14.879117 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.879079    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q282g\" (UniqueName: \"kubernetes.io/projected/50eb7188-4911-46f1-9bd1-ececaa36fd1c-kube-api-access-q282g\") pod \"test-curl-7831-140710496211136\" (UID: \"50eb7188-4911-46f1-9bd1-ececaa36fd1c\") " pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:14.979984 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.979901    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q282g\" (UniqueName: \"kubernetes.io/projected/50eb7188-4911-46f1-9bd1-ececaa36fd1c-kube-api-access-q282g\") pod \"test-curl-7831-140710496211136\" (UID: \"50eb7188-4911-46f1-9bd1-ececaa36fd1c\") " pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:14.989295 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:14.989270    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q282g\" (UniqueName: \"kubernetes.io/projected/50eb7188-4911-46f1-9bd1-ececaa36fd1c-kube-api-access-q282g\") pod \"test-curl-7831-140710496211136\" (UID: \"50eb7188-4911-46f1-9bd1-ececaa36fd1c\") " pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:15.068779 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:15.068748    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:15.208160 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:15.208116    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:15.209960 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:14:15.209934    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50eb7188_4911_46f1_9bd1_ececaa36fd1c.slice/crio-b99e068aa1b766726fa045a77e6984bb3785a81c7d02e56918f25104e1fe12ce WatchSource:0}: Error finding container b99e068aa1b766726fa045a77e6984bb3785a81c7d02e56918f25104e1fe12ce: Status 404 returned error can't find the container with id b99e068aa1b766726fa045a77e6984bb3785a81c7d02e56918f25104e1fe12ce
Jul 29 16:14:15.397013 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:15.396974    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:15.397013 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:15.397004    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3dd6b872f31a34ab52395e6406bbdc78591543b066247d93c68b9352712f7134"
Jul 29 16:14:15.398172 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:15.398142    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496211136" event={"ID":"50eb7188-4911-46f1-9bd1-ececaa36fd1c","Type":"ContainerStarted","Data":"b99e068aa1b766726fa045a77e6984bb3785a81c7d02e56918f25104e1fe12ce"}
Jul 29 16:14:15.407889 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:14:15.407858    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7831-140710491538864\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="c23c5063-0dce-402d-adbe-f4a95dc952bf" pod="opendatahub/test-curl-7831-140710491538864"
Jul 29 16:14:16.167326 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:16.167291    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c23c5063-0dce-402d-adbe-f4a95dc952bf" path="/var/lib/kubelet/pods/c23c5063-0dce-402d-adbe-f4a95dc952bf/volumes"
Jul 29 16:14:16.405547 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:16.405512    2560 generic.go:356] "Generic (PLEG): container finished" podID="50eb7188-4911-46f1-9bd1-ececaa36fd1c" containerID="79e45c26f696583157d42b808477b858c3e03f4b17c169be3dad5c5f271617c7" exitCode=0
Jul 29 16:14:16.405739 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:16.405592    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496211136" event={"ID":"50eb7188-4911-46f1-9bd1-ececaa36fd1c","Type":"ContainerDied","Data":"79e45c26f696583157d42b808477b858c3e03f4b17c169be3dad5c5f271617c7"}
Jul 29 16:14:17.552716 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.552684    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:17.576572 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.576517    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7831-140710496211136_50eb7188-4911-46f1-9bd1-ececaa36fd1c/test-curl-7831-140710496211136/0.log"
Jul 29 16:14:17.604346 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.604305    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:17.604579 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.604556    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q282g\" (UniqueName: \"kubernetes.io/projected/50eb7188-4911-46f1-9bd1-ececaa36fd1c-kube-api-access-q282g\") pod \"50eb7188-4911-46f1-9bd1-ececaa36fd1c\" (UID: \"50eb7188-4911-46f1-9bd1-ececaa36fd1c\") "
Jul 29 16:14:17.606693 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.606666    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50eb7188-4911-46f1-9bd1-ececaa36fd1c-kube-api-access-q282g" (OuterVolumeSpecName: "kube-api-access-q282g") pod "50eb7188-4911-46f1-9bd1-ececaa36fd1c" (UID: "50eb7188-4911-46f1-9bd1-ececaa36fd1c"). InnerVolumeSpecName "kube-api-access-q282g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:17.611240 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.611205    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:17.705724 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.705632    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q282g\" (UniqueName: \"kubernetes.io/projected/50eb7188-4911-46f1-9bd1-ececaa36fd1c-kube-api-access-q282g\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:17.736170 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.736118    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:17.736536 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.736520    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50eb7188-4911-46f1-9bd1-ececaa36fd1c" containerName="test-curl-7831-140710496211136"
Jul 29 16:14:17.736536 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.736537    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="50eb7188-4911-46f1-9bd1-ececaa36fd1c" containerName="test-curl-7831-140710496211136"
Jul 29 16:14:17.736630 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.736613    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="50eb7188-4911-46f1-9bd1-ececaa36fd1c" containerName="test-curl-7831-140710496211136"
Jul 29 16:14:17.739697 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.739677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:17.746679 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.746652    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:17.806257 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.806213    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn5b8\" (UniqueName: \"kubernetes.io/projected/c74db66b-e298-4bc6-b671-833c0394d002-kube-api-access-mn5b8\") pod \"test-curl-7831-140710496211136\" (UID: \"c74db66b-e298-4bc6-b671-833c0394d002\") " pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:17.907330 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.907286    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn5b8\" (UniqueName: \"kubernetes.io/projected/c74db66b-e298-4bc6-b671-833c0394d002-kube-api-access-mn5b8\") pod \"test-curl-7831-140710496211136\" (UID: \"c74db66b-e298-4bc6-b671-833c0394d002\") " pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:17.916665 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:17.916634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn5b8\" (UniqueName: \"kubernetes.io/projected/c74db66b-e298-4bc6-b671-833c0394d002-kube-api-access-mn5b8\") pod \"test-curl-7831-140710496211136\" (UID: \"c74db66b-e298-4bc6-b671-833c0394d002\") " pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:18.049870 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:18.049774    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:18.168951 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:18.168915    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50eb7188-4911-46f1-9bd1-ececaa36fd1c" path="/var/lib/kubelet/pods/50eb7188-4911-46f1-9bd1-ececaa36fd1c/volumes"
Jul 29 16:14:18.187985 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:18.187957    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:18.190620 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:14:18.190590    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc74db66b_e298_4bc6_b671_833c0394d002.slice/crio-f2b6f99c2a32cd7b761f7dac6acf0b73516b4a0e33c0fadb4c1b4e6fc1805aeb WatchSource:0}: Error finding container f2b6f99c2a32cd7b761f7dac6acf0b73516b4a0e33c0fadb4c1b4e6fc1805aeb: Status 404 returned error can't find the container with id f2b6f99c2a32cd7b761f7dac6acf0b73516b4a0e33c0fadb4c1b4e6fc1805aeb
Jul 29 16:14:18.420039 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:18.419988    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496211136" event={"ID":"c74db66b-e298-4bc6-b671-833c0394d002","Type":"ContainerStarted","Data":"f2b6f99c2a32cd7b761f7dac6acf0b73516b4a0e33c0fadb4c1b4e6fc1805aeb"}
Jul 29 16:14:18.421230 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:18.421210    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:18.421340 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:18.421240    2560 scope.go:117] "RemoveContainer" containerID="79e45c26f696583157d42b808477b858c3e03f4b17c169be3dad5c5f271617c7"
Jul 29 16:14:19.428135 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:19.428096    2560 generic.go:356] "Generic (PLEG): container finished" podID="c74db66b-e298-4bc6-b671-833c0394d002" containerID="0af28829bd8d5968cbefca1d7b325df631b186a261f113d27e3b781160139446" exitCode=0
Jul 29 16:14:19.428646 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:19.428212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496211136" event={"ID":"c74db66b-e298-4bc6-b671-833c0394d002","Type":"ContainerDied","Data":"0af28829bd8d5968cbefca1d7b325df631b186a261f113d27e3b781160139446"}
Jul 29 16:14:20.581569 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.581540    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:20.607783 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.607734    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7831-140710496211136_c74db66b-e298-4bc6-b671-833c0394d002/test-curl-7831-140710496211136/0.log"
Jul 29 16:14:20.630958 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.630916    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn5b8\" (UniqueName: \"kubernetes.io/projected/c74db66b-e298-4bc6-b671-833c0394d002-kube-api-access-mn5b8\") pod \"c74db66b-e298-4bc6-b671-833c0394d002\" (UID: \"c74db66b-e298-4bc6-b671-833c0394d002\") "
Jul 29 16:14:20.633082 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.633054    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c74db66b-e298-4bc6-b671-833c0394d002-kube-api-access-mn5b8" (OuterVolumeSpecName: "kube-api-access-mn5b8") pod "c74db66b-e298-4bc6-b671-833c0394d002" (UID: "c74db66b-e298-4bc6-b671-833c0394d002"). InnerVolumeSpecName "kube-api-access-mn5b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:20.636330 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.636301    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:20.641251 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.641228    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7831-140710496211136"]
Jul 29 16:14:20.731685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.731595    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn5b8\" (UniqueName: \"kubernetes.io/projected/c74db66b-e298-4bc6-b671-833c0394d002-kube-api-access-mn5b8\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:20.765056 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.765019    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:20.765457 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.765442    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74db66b-e298-4bc6-b671-833c0394d002" containerName="test-curl-7831-140710496211136"
Jul 29 16:14:20.765521 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.765459    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74db66b-e298-4bc6-b671-833c0394d002" containerName="test-curl-7831-140710496211136"
Jul 29 16:14:20.765560 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.765525    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c74db66b-e298-4bc6-b671-833c0394d002" containerName="test-curl-7831-140710496211136"
Jul 29 16:14:20.768559 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.768538    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:20.773798 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.773766    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:20.832435 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.832395    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td7rw\" (UniqueName: \"kubernetes.io/projected/7870a239-12d5-488f-bc6b-57c21faf049d-kube-api-access-td7rw\") pod \"test-curl-7831-140710496212144\" (UID: \"7870a239-12d5-488f-bc6b-57c21faf049d\") " pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:20.933394 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.933362    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td7rw\" (UniqueName: \"kubernetes.io/projected/7870a239-12d5-488f-bc6b-57c21faf049d-kube-api-access-td7rw\") pod \"test-curl-7831-140710496212144\" (UID: \"7870a239-12d5-488f-bc6b-57c21faf049d\") " pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:20.943090 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:20.943062    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td7rw\" (UniqueName: \"kubernetes.io/projected/7870a239-12d5-488f-bc6b-57c21faf049d-kube-api-access-td7rw\") pod \"test-curl-7831-140710496212144\" (UID: \"7870a239-12d5-488f-bc6b-57c21faf049d\") " pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:21.079640 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.079552    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:21.211655 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.211613    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:21.214443 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:14:21.214419    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7870a239_12d5_488f_bc6b_57c21faf049d.slice/crio-45bdd8f59886772283ebe7ae4fcd5020b04803fd1f93094ebe14c2061eb414ad WatchSource:0}: Error finding container 45bdd8f59886772283ebe7ae4fcd5020b04803fd1f93094ebe14c2061eb414ad: Status 404 returned error can't find the container with id 45bdd8f59886772283ebe7ae4fcd5020b04803fd1f93094ebe14c2061eb414ad
Jul 29 16:14:21.348682 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.348636    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw" podUID="24846591-f8fe-4edb-9e14-9601c0fc902a" containerName="main" containerID="cri-o://ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4" gracePeriod=45
Jul 29 16:14:21.441912 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.441878    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496212144" event={"ID":"7870a239-12d5-488f-bc6b-57c21faf049d","Type":"ContainerStarted","Data":"45bdd8f59886772283ebe7ae4fcd5020b04803fd1f93094ebe14c2061eb414ad"}
Jul 29 16:14:21.443073 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.443049    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2b6f99c2a32cd7b761f7dac6acf0b73516b4a0e33c0fadb4c1b4e6fc1805aeb"
Jul 29 16:14:21.443219 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.443080    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:21.456898 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:14:21.456859    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7831-140710496211136\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="c74db66b-e298-4bc6-b671-833c0394d002" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:21.662545 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.662514    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:14:21.683618 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:14:21.683570    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7831-140710496211136\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="c74db66b-e298-4bc6-b671-833c0394d002" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:21.742953 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.742921    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-model-cache\") pod \"24846591-f8fe-4edb-9e14-9601c0fc902a\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") "
Jul 29 16:14:21.742953 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.742956    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l22v\" (UniqueName: \"kubernetes.io/projected/24846591-f8fe-4edb-9e14-9601c0fc902a-kube-api-access-6l22v\") pod \"24846591-f8fe-4edb-9e14-9601c0fc902a\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") "
Jul 29 16:14:21.743221 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743028    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-dshm\") pod \"24846591-f8fe-4edb-9e14-9601c0fc902a\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") "
Jul 29 16:14:21.743221 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743077    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24846591-f8fe-4edb-9e14-9601c0fc902a-tls-certs\") pod \"24846591-f8fe-4edb-9e14-9601c0fc902a\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") "
Jul 29 16:14:21.743221 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743103    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-tmp-dir\") pod \"24846591-f8fe-4edb-9e14-9601c0fc902a\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") "
Jul 29 16:14:21.743221 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743205    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-home\") pod \"24846591-f8fe-4edb-9e14-9601c0fc902a\" (UID: \"24846591-f8fe-4edb-9e14-9601c0fc902a\") "
Jul 29 16:14:21.743443 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743226    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-model-cache" (OuterVolumeSpecName: "model-cache") pod "24846591-f8fe-4edb-9e14-9601c0fc902a" (UID: "24846591-f8fe-4edb-9e14-9601c0fc902a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:21.743443 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743408    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "24846591-f8fe-4edb-9e14-9601c0fc902a" (UID: "24846591-f8fe-4edb-9e14-9601c0fc902a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:21.743533 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743514    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-home" (OuterVolumeSpecName: "home") pod "24846591-f8fe-4edb-9e14-9601c0fc902a" (UID: "24846591-f8fe-4edb-9e14-9601c0fc902a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:21.743572 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743539    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:21.743572 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.743553    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:21.745329 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.745301    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24846591-f8fe-4edb-9e14-9601c0fc902a-kube-api-access-6l22v" (OuterVolumeSpecName: "kube-api-access-6l22v") pod "24846591-f8fe-4edb-9e14-9601c0fc902a" (UID: "24846591-f8fe-4edb-9e14-9601c0fc902a"). InnerVolumeSpecName "kube-api-access-6l22v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:21.745329 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.745319    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24846591-f8fe-4edb-9e14-9601c0fc902a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "24846591-f8fe-4edb-9e14-9601c0fc902a" (UID: "24846591-f8fe-4edb-9e14-9601c0fc902a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:14:21.745475 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.745429    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-dshm" (OuterVolumeSpecName: "dshm") pod "24846591-f8fe-4edb-9e14-9601c0fc902a" (UID: "24846591-f8fe-4edb-9e14-9601c0fc902a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:21.844880 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.844837    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:21.844880 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.844869    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24846591-f8fe-4edb-9e14-9601c0fc902a-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:21.844880 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.844880    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24846591-f8fe-4edb-9e14-9601c0fc902a-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:21.844880 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:21.844889    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l22v\" (UniqueName: \"kubernetes.io/projected/24846591-f8fe-4edb-9e14-9601c0fc902a-kube-api-access-6l22v\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:22.165431 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:14:22.165392    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7831-140710496211136\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="c74db66b-e298-4bc6-b671-833c0394d002" pod="opendatahub/test-curl-7831-140710496211136"
Jul 29 16:14:22.166353 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.166332    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c74db66b-e298-4bc6-b671-833c0394d002" path="/var/lib/kubelet/pods/c74db66b-e298-4bc6-b671-833c0394d002/volumes"
Jul 29 16:14:22.414226 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.414160    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp" podUID="c77e6779-334d-4d78-a133-74fe54a08cca" containerName="main" containerID="cri-o://5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84" gracePeriod=45
Jul 29 16:14:22.450083 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.450055    2560 generic.go:356] "Generic (PLEG): container finished" podID="7870a239-12d5-488f-bc6b-57c21faf049d" containerID="833922ffb85a64e4380c373bfa225818aabc61b5b59d91bb6157c2a1fba7ef3a" exitCode=0
Jul 29 16:14:22.450252 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.450145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496212144" event={"ID":"7870a239-12d5-488f-bc6b-57c21faf049d","Type":"ContainerDied","Data":"833922ffb85a64e4380c373bfa225818aabc61b5b59d91bb6157c2a1fba7ef3a"}
Jul 29 16:14:22.451498 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.451474    2560 generic.go:356] "Generic (PLEG): container finished" podID="24846591-f8fe-4edb-9e14-9601c0fc902a" containerID="ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4" exitCode=0
Jul 29 16:14:22.451612 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.451511    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"
Jul 29 16:14:22.451612 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.451525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw" event={"ID":"24846591-f8fe-4edb-9e14-9601c0fc902a","Type":"ContainerDied","Data":"ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4"}
Jul 29 16:14:22.451612 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.451554    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw" event={"ID":"24846591-f8fe-4edb-9e14-9601c0fc902a","Type":"ContainerDied","Data":"21a6c283f8b42829ac89bc089927a6e562a15db6228b25043160d6b85ca776f2"}
Jul 29 16:14:22.451612 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.451571    2560 scope.go:117] "RemoveContainer" containerID="ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4"
Jul 29 16:14:22.488368 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.488330    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"]
Jul 29 16:14:22.493735 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.493708    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-02add2/rate-test-model-02add2-kserve-5c6695c6bc-gb6qw"]
Jul 29 16:14:22.526299 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.526119    2560 scope.go:117] "RemoveContainer" containerID="ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4"
Jul 29 16:14:22.526512 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:14:22.526490    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4\": container with ID starting with ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4 not found: ID does not exist" containerID="ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4"
Jul 29 16:14:22.526550 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.526524    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4"} err="failed to get container status \"ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4\": rpc error: code = NotFound desc = could not find container \"ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4\": container with ID starting with ecfdd6600084fd953f39f88dae56d0d739ec3bdd230e2f9c86044c591053c4c4 not found: ID does not exist"
Jul 29 16:14:22.673012 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.672949    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:14:22.752429 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752399    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/c77e6779-334d-4d78-a133-74fe54a08cca-kube-api-access-l47qz\") pod \"c77e6779-334d-4d78-a133-74fe54a08cca\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") "
Jul 29 16:14:22.752591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752441    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77e6779-334d-4d78-a133-74fe54a08cca-tls-certs\") pod \"c77e6779-334d-4d78-a133-74fe54a08cca\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") "
Jul 29 16:14:22.752591 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752474    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-tmp-dir\") pod \"c77e6779-334d-4d78-a133-74fe54a08cca\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") "
Jul 29 16:14:22.752670 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752608    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-dshm\") pod \"c77e6779-334d-4d78-a133-74fe54a08cca\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") "
Jul 29 16:14:22.752670 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752636    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-home\") pod \"c77e6779-334d-4d78-a133-74fe54a08cca\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") "
Jul 29 16:14:22.752670 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752657    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-model-cache\") pod \"c77e6779-334d-4d78-a133-74fe54a08cca\" (UID: \"c77e6779-334d-4d78-a133-74fe54a08cca\") "
Jul 29 16:14:22.752844 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752809    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c77e6779-334d-4d78-a133-74fe54a08cca" (UID: "c77e6779-334d-4d78-a133-74fe54a08cca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:22.752906 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752891    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-home" (OuterVolumeSpecName: "home") pod "c77e6779-334d-4d78-a133-74fe54a08cca" (UID: "c77e6779-334d-4d78-a133-74fe54a08cca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:22.753013 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752985    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-model-cache" (OuterVolumeSpecName: "model-cache") pod "c77e6779-334d-4d78-a133-74fe54a08cca" (UID: "c77e6779-334d-4d78-a133-74fe54a08cca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:22.753013 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.752998    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:22.753154 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.753034    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:22.754629 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.754598    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-dshm" (OuterVolumeSpecName: "dshm") pod "c77e6779-334d-4d78-a133-74fe54a08cca" (UID: "c77e6779-334d-4d78-a133-74fe54a08cca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:14:22.754734 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.754667    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c77e6779-334d-4d78-a133-74fe54a08cca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c77e6779-334d-4d78-a133-74fe54a08cca" (UID: "c77e6779-334d-4d78-a133-74fe54a08cca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:14:22.754734 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.754687    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77e6779-334d-4d78-a133-74fe54a08cca-kube-api-access-l47qz" (OuterVolumeSpecName: "kube-api-access-l47qz") pod "c77e6779-334d-4d78-a133-74fe54a08cca" (UID: "c77e6779-334d-4d78-a133-74fe54a08cca"). InnerVolumeSpecName "kube-api-access-l47qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:22.853965 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.853922    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:22.853965 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.853960    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c77e6779-334d-4d78-a133-74fe54a08cca-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:22.853965 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.853971    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/c77e6779-334d-4d78-a133-74fe54a08cca-kube-api-access-l47qz\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:22.853965 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:22.853980    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c77e6779-334d-4d78-a133-74fe54a08cca-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:23.459806 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.459761    2560 generic.go:356] "Generic (PLEG): container finished" podID="c77e6779-334d-4d78-a133-74fe54a08cca" containerID="5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84" exitCode=0
Jul 29 16:14:23.459806 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.459795    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp" event={"ID":"c77e6779-334d-4d78-a133-74fe54a08cca","Type":"ContainerDied","Data":"5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84"}
Jul 29 16:14:23.460079 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.459844    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp" event={"ID":"c77e6779-334d-4d78-a133-74fe54a08cca","Type":"ContainerDied","Data":"2d6c46f7ca9c7454fcc9a92ea7a7471099ef4a433419a6e023610d10bae7424b"}
Jul 29 16:14:23.460079 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.459868    2560 scope.go:117] "RemoveContainer" containerID="5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84"
Jul 29 16:14:23.460079 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.459811    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"
Jul 29 16:14:23.469801 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.469776    2560 scope.go:117] "RemoveContainer" containerID="5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84"
Jul 29 16:14:23.470131 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:14:23.470114    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84\": container with ID starting with 5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84 not found: ID does not exist" containerID="5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84"
Jul 29 16:14:23.470253 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.470139    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84"} err="failed to get container status \"5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84\": rpc error: code = NotFound desc = could not find container \"5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84\": container with ID starting with 5e62b4ad1a7aa008078f882feb6fe7c616ca5fcde3478fd57135d89943cf9e84 not found: ID does not exist"
Jul 29 16:14:23.490120 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.490084    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"]
Jul 29 16:14:23.494463 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.494435    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-7e5daf/rate-test-model-7e5daf-kserve-545cc768b7-zz9jp"]
Jul 29 16:14:23.612477 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.612450    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:23.635606 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.635577    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7831-140710496212144_7870a239-12d5-488f-bc6b-57c21faf049d/test-curl-7831-140710496212144/0.log"
Jul 29 16:14:23.662148 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.661781    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td7rw\" (UniqueName: \"kubernetes.io/projected/7870a239-12d5-488f-bc6b-57c21faf049d-kube-api-access-td7rw\") pod \"7870a239-12d5-488f-bc6b-57c21faf049d\" (UID: \"7870a239-12d5-488f-bc6b-57c21faf049d\") "
Jul 29 16:14:23.663076 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.663052    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:23.664039 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.664016    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7870a239-12d5-488f-bc6b-57c21faf049d-kube-api-access-td7rw" (OuterVolumeSpecName: "kube-api-access-td7rw") pod "7870a239-12d5-488f-bc6b-57c21faf049d" (UID: "7870a239-12d5-488f-bc6b-57c21faf049d"). InnerVolumeSpecName "kube-api-access-td7rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:23.665949 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.665919    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:23.762668 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.762579    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td7rw\" (UniqueName: \"kubernetes.io/projected/7870a239-12d5-488f-bc6b-57c21faf049d-kube-api-access-td7rw\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:23.790201 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790153    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:23.790548 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790535    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24846591-f8fe-4edb-9e14-9601c0fc902a" containerName="main"
Jul 29 16:14:23.790651 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790550    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="24846591-f8fe-4edb-9e14-9601c0fc902a" containerName="main"
Jul 29 16:14:23.790651 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790562    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7870a239-12d5-488f-bc6b-57c21faf049d" containerName="test-curl-7831-140710496212144"
Jul 29 16:14:23.790651 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790568    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7870a239-12d5-488f-bc6b-57c21faf049d" containerName="test-curl-7831-140710496212144"
Jul 29 16:14:23.790651 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790581    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77e6779-334d-4d78-a133-74fe54a08cca" containerName="main"
Jul 29 16:14:23.790651 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790586    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77e6779-334d-4d78-a133-74fe54a08cca" containerName="main"
Jul 29 16:14:23.790651 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790641    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="24846591-f8fe-4edb-9e14-9601c0fc902a" containerName="main"
Jul 29 16:14:23.790651 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790651    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77e6779-334d-4d78-a133-74fe54a08cca" containerName="main"
Jul 29 16:14:23.790860 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.790659    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7870a239-12d5-488f-bc6b-57c21faf049d" containerName="test-curl-7831-140710496212144"
Jul 29 16:14:23.793684 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.793667    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:23.799577 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.799545    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:23.863238 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.863204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz4pb\" (UniqueName: \"kubernetes.io/projected/861aefa8-f285-45a7-aa83-566859323631-kube-api-access-kz4pb\") pod \"test-curl-7831-140710496212144\" (UID: \"861aefa8-f285-45a7-aa83-566859323631\") " pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:23.964166 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.964122    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz4pb\" (UniqueName: \"kubernetes.io/projected/861aefa8-f285-45a7-aa83-566859323631-kube-api-access-kz4pb\") pod \"test-curl-7831-140710496212144\" (UID: \"861aefa8-f285-45a7-aa83-566859323631\") " pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:23.974563 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:23.974538    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz4pb\" (UniqueName: \"kubernetes.io/projected/861aefa8-f285-45a7-aa83-566859323631-kube-api-access-kz4pb\") pod \"test-curl-7831-140710496212144\" (UID: \"861aefa8-f285-45a7-aa83-566859323631\") " pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:24.105382 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.105342    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:24.168723 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.168691    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24846591-f8fe-4edb-9e14-9601c0fc902a" path="/var/lib/kubelet/pods/24846591-f8fe-4edb-9e14-9601c0fc902a/volumes"
Jul 29 16:14:24.169067 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.169051    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7870a239-12d5-488f-bc6b-57c21faf049d" path="/var/lib/kubelet/pods/7870a239-12d5-488f-bc6b-57c21faf049d/volumes"
Jul 29 16:14:24.169458 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.169444    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c77e6779-334d-4d78-a133-74fe54a08cca" path="/var/lib/kubelet/pods/c77e6779-334d-4d78-a133-74fe54a08cca/volumes"
Jul 29 16:14:24.243552 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.243524    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:24.245205 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:14:24.245152    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod861aefa8_f285_45a7_aa83_566859323631.slice/crio-8f7819ef90bd767834214d07a9023792e525b9ef39dda366be8be77f34092f75 WatchSource:0}: Error finding container 8f7819ef90bd767834214d07a9023792e525b9ef39dda366be8be77f34092f75: Status 404 returned error can't find the container with id 8f7819ef90bd767834214d07a9023792e525b9ef39dda366be8be77f34092f75
Jul 29 16:14:24.474433 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.474338    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496212144" event={"ID":"861aefa8-f285-45a7-aa83-566859323631","Type":"ContainerStarted","Data":"8f7819ef90bd767834214d07a9023792e525b9ef39dda366be8be77f34092f75"}
Jul 29 16:14:24.475495 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.475473    2560 scope.go:117] "RemoveContainer" containerID="833922ffb85a64e4380c373bfa225818aabc61b5b59d91bb6157c2a1fba7ef3a"
Jul 29 16:14:24.475495 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:24.475482    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:25.483004 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:25.482969    2560 generic.go:356] "Generic (PLEG): container finished" podID="861aefa8-f285-45a7-aa83-566859323631" containerID="ab4e5c51cdf42e86f06af39e45983e5388b26ea667cac119e3c30401ae138967" exitCode=0
Jul 29 16:14:25.483453 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:25.483054    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7831-140710496212144" event={"ID":"861aefa8-f285-45a7-aa83-566859323631","Type":"ContainerDied","Data":"ab4e5c51cdf42e86f06af39e45983e5388b26ea667cac119e3c30401ae138967"}
Jul 29 16:14:26.626358 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:26.626334    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:26.649728 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:26.649697    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7831-140710496212144_861aefa8-f285-45a7-aa83-566859323631/test-curl-7831-140710496212144/0.log"
Jul 29 16:14:26.678924 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:26.678890    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:26.685986 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:26.685958    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7831-140710496212144"]
Jul 29 16:14:26.688009 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:26.687978    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz4pb\" (UniqueName: \"kubernetes.io/projected/861aefa8-f285-45a7-aa83-566859323631-kube-api-access-kz4pb\") pod \"861aefa8-f285-45a7-aa83-566859323631\" (UID: \"861aefa8-f285-45a7-aa83-566859323631\") "
Jul 29 16:14:26.690041 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:26.690018    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861aefa8-f285-45a7-aa83-566859323631-kube-api-access-kz4pb" (OuterVolumeSpecName: "kube-api-access-kz4pb") pod "861aefa8-f285-45a7-aa83-566859323631" (UID: "861aefa8-f285-45a7-aa83-566859323631"). InnerVolumeSpecName "kube-api-access-kz4pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:14:26.789060 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:26.788961    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz4pb\" (UniqueName: \"kubernetes.io/projected/861aefa8-f285-45a7-aa83-566859323631-kube-api-access-kz4pb\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:14:27.497226 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:27.497175    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f7819ef90bd767834214d07a9023792e525b9ef39dda366be8be77f34092f75"
Jul 29 16:14:27.497415 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:27.497240    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:27.509849 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:14:27.509814    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7831-140710496212144\" is forbidden: User \"system:node:ip-10-0-136-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-199.ec2.internal' and this object" podUID="861aefa8-f285-45a7-aa83-566859323631" pod="opendatahub/test-curl-7831-140710496212144"
Jul 29 16:14:28.166263 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:28.166227    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861aefa8-f285-45a7-aa83-566859323631" path="/var/lib/kubelet/pods/861aefa8-f285-45a7-aa83-566859323631/volumes"
Jul 29 16:14:31.481785 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.481748    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"]
Jul 29 16:14:31.482250 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.482127    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861aefa8-f285-45a7-aa83-566859323631" containerName="test-curl-7831-140710496212144"
Jul 29 16:14:31.482250 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.482144    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="861aefa8-f285-45a7-aa83-566859323631" containerName="test-curl-7831-140710496212144"
Jul 29 16:14:31.482343 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.482263    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="861aefa8-f285-45a7-aa83-566859323631" containerName="test-curl-7831-140710496212144"
Jul 29 16:14:31.487417 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.487395    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.489801 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.489783    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-6fccfc\""
Jul 29 16:14:31.489936 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.489800    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-6fccfc-dockercfg-7mt95\""
Jul 29 16:14:31.500795 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.500768    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"]
Jul 29 16:14:31.641258 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.641214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6z7q\" (UniqueName: \"kubernetes.io/projected/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-kube-api-access-r6z7q\") pod \"payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.641427 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.641275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.742587 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.742492    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6z7q\" (UniqueName: \"kubernetes.io/projected/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-kube-api-access-r6z7q\") pod \"payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.742587 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.742559    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.743263 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.743244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.752249 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.752226    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6z7q\" (UniqueName: \"kubernetes.io/projected/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-kube-api-access-r6z7q\") pod \"payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.798777 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.798735    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:31.942356 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:31.942326    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"]
Jul 29 16:14:31.945244 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:14:31.945216    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf87d4a5_1bc4_49ad_aaff_a3625cc879a0.slice/crio-a119fb74514a3cecb71741f7db827d3af8138882905a4ea00e6a95657d735cb3 WatchSource:0}: Error finding container a119fb74514a3cecb71741f7db827d3af8138882905a4ea00e6a95657d735cb3: Status 404 returned error can't find the container with id a119fb74514a3cecb71741f7db827d3af8138882905a4ea00e6a95657d735cb3
Jul 29 16:14:32.526526 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:32.526488    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v" event={"ID":"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0","Type":"ContainerStarted","Data":"a7d80e4de8e0084b6d39f74abf00ddcff1d9487f47cdad063e3c1fca7e18dc64"}
Jul 29 16:14:32.526942 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:32.526645    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:14:32.526942 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:32.526665    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v" event={"ID":"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0","Type":"ContainerStarted","Data":"a119fb74514a3cecb71741f7db827d3af8138882905a4ea00e6a95657d735cb3"}
Jul 29 16:14:32.547196 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:32.547145    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v" podStartSLOduration=1.547130578 podStartE2EDuration="1.547130578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:14:32.54535364 +0000 UTC m=+3810.839290442" watchObservedRunningTime="2026-07-29 16:14:32.547130578 +0000 UTC m=+3810.841067379"
Jul 29 16:14:39.582320 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:39.582284    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:14:43.535026 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:14:43.534995    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:15:00.156014 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.155970    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"]
Jul 29 16:15:00.159533 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.159512    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt"]
Jul 29 16:15:00.159652 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.159633    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"
Jul 29 16:15:00.162559 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.162535    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:15:00.162929 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.162908    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt"
Jul 29 16:15:00.173962 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.173933    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"]
Jul 29 16:15:00.176115 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.176090    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"
Jul 29 16:15:00.178126 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.178104    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"]
Jul 29 16:15:00.178430 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.178232    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"
Jul 29 16:15:00.186151 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.186125    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"]
Jul 29 16:15:00.186304 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.186271    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"
Jul 29 16:15:00.187299 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.187279    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt"
Jul 29 16:15:00.192354 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.190643    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt"]
Jul 29 16:15:00.192354 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.190686    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"]
Jul 29 16:15:00.192354 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.190709    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"
Jul 29 16:15:00.192354 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.190713    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"]
Jul 29 16:15:00.192354 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.190846    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"]
Jul 29 16:15:00.192354 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.192291    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"]
Jul 29 16:15:00.234458 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.234391    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"
Jul 29 16:15:00.248686 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.248468    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"
Jul 29 16:15:00.287804 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.287777    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"
Jul 29 16:15:00.381123 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.380641    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"]
Jul 29 16:15:00.616936 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.616895    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"]
Jul 29 16:15:00.624446 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.624420    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:00.624875 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.624842    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt"]
Jul 29 16:15:00.626790 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:15:00.626618    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405fda58_82b9_4d07_9509_75fff1df213e.slice/crio-99dc743f440736dcafe4c3ddd6433ab78c24bf8faf7d0a44ba2e1a6d0dc2ab31 WatchSource:0}: Error finding container 99dc743f440736dcafe4c3ddd6433ab78c24bf8faf7d0a44ba2e1a6d0dc2ab31: Status 404 returned error can't find the container with id 99dc743f440736dcafe4c3ddd6433ab78c24bf8faf7d0a44ba2e1a6d0dc2ab31
Jul 29 16:15:00.628213 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.628162    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-16a6d7\""
Jul 29 16:15:00.630230 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.630206    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"]
Jul 29 16:15:00.678726 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.678688    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"]
Jul 29 16:15:00.679279 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:15:00.679252    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdf705a_ec3f_4e80_8d06_32767ef46863.slice/crio-e777a19c34daca4d45ce0e704a43cb1865f1716d3a81e64f1c6a81bbcf0d5715 WatchSource:0}: Error finding container e777a19c34daca4d45ce0e704a43cb1865f1716d3a81e64f1c6a81bbcf0d5715: Status 404 returned error can't find the container with id e777a19c34daca4d45ce0e704a43cb1865f1716d3a81e64f1c6a81bbcf0d5715
Jul 29 16:15:00.686260 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.686227    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt" event={"ID":"405fda58-82b9-4d07-9509-75fff1df213e","Type":"ContainerStarted","Data":"99dc743f440736dcafe4c3ddd6433ab78c24bf8faf7d0a44ba2e1a6d0dc2ab31"}
Jul 29 16:15:00.687723 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.687694    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw" event={"ID":"5b81cb20-dcd1-416a-ad56-3963d3dac510","Type":"ContainerStarted","Data":"69d527c12cc2dde6db507adcee0ae652831c8c8ad19acdd826e9b3fecc7983e9"}
Jul 29 16:15:00.687822 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.687753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw" event={"ID":"5b81cb20-dcd1-416a-ad56-3963d3dac510","Type":"ContainerStarted","Data":"58c4763caf401abfeb5c4f44175d5363cdafc48277f6018096aaa098ffd6bbad"}
Jul 29 16:15:00.699558 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.699531    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"]
Jul 29 16:15:00.700601 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:15:00.700575    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cbd4336_7cc1_4f8e_86c4_f9dce4393b8b.slice/crio-c40c3d61510bb911380fb0dcfd6f2233de232a27856e06aabb8d8dc54c871073 WatchSource:0}: Error finding container c40c3d61510bb911380fb0dcfd6f2233de232a27856e06aabb8d8dc54c871073: Status 404 returned error can't find the container with id c40c3d61510bb911380fb0dcfd6f2233de232a27856e06aabb8d8dc54c871073
Jul 29 16:15:00.709656 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.709478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458c6823-2a62-4357-8f2d-b798e43529ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:00.709656 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.709548    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqjlt\" (UniqueName: \"kubernetes.io/projected/458c6823-2a62-4357-8f2d-b798e43529ad-kube-api-access-fqjlt\") pod \"payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:00.711859 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.711812    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw" podStartSLOduration=0.711797216 podStartE2EDuration="711.797216ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:00.708922931 +0000 UTC m=+3839.002859736" watchObservedRunningTime="2026-07-29 16:15:00.711797216 +0000 UTC m=+3839.005734052"
Jul 29 16:15:00.725242 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.725215    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"]
Jul 29 16:15:00.737418 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:15:00.737388    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c0d42a_5e11_4f8f_b12e_29d59572a561.slice/crio-c7d7ef0547e4e3df458610a15d6e3b788a23904f472152900477efca7ab70693 WatchSource:0}: Error finding container c7d7ef0547e4e3df458610a15d6e3b788a23904f472152900477efca7ab70693: Status 404 returned error can't find the container with id c7d7ef0547e4e3df458610a15d6e3b788a23904f472152900477efca7ab70693
Jul 29 16:15:00.810534 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.810493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458c6823-2a62-4357-8f2d-b798e43529ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:00.810724 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.810556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqjlt\" (UniqueName: \"kubernetes.io/projected/458c6823-2a62-4357-8f2d-b798e43529ad-kube-api-access-fqjlt\") pod \"payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:00.811243 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.811222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458c6823-2a62-4357-8f2d-b798e43529ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:00.842776 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.842744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqjlt\" (UniqueName: \"kubernetes.io/projected/458c6823-2a62-4357-8f2d-b798e43529ad-kube-api-access-fqjlt\") pod \"payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:00.938205 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:00.938105    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:01.140277 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.140242    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"]
Jul 29 16:15:01.142350 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:15:01.142319    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458c6823_2a62_4357_8f2d_b798e43529ad.slice/crio-9a4225730c02d660b7772acef6ce234f536e1f3ec896463af24cfe12b9e430ce WatchSource:0}: Error finding container 9a4225730c02d660b7772acef6ce234f536e1f3ec896463af24cfe12b9e430ce: Status 404 returned error can't find the container with id 9a4225730c02d660b7772acef6ce234f536e1f3ec896463af24cfe12b9e430ce
Jul 29 16:15:01.695851 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.695751    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" event={"ID":"54c0d42a-5e11-4f8f-b12e-29d59572a561","Type":"ContainerStarted","Data":"4552d314f05cb7506725c50f9b47f7b0e6dcd2b93183cd3d2b20289a2784a5c0"}
Jul 29 16:15:01.695851 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.695817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" event={"ID":"54c0d42a-5e11-4f8f-b12e-29d59572a561","Type":"ContainerStarted","Data":"c7d7ef0547e4e3df458610a15d6e3b788a23904f472152900477efca7ab70693"}
Jul 29 16:15:01.697484 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.697457    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4" event={"ID":"458c6823-2a62-4357-8f2d-b798e43529ad","Type":"ContainerStarted","Data":"6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b"}
Jul 29 16:15:01.697645 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.697499    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4" event={"ID":"458c6823-2a62-4357-8f2d-b798e43529ad","Type":"ContainerStarted","Data":"9a4225730c02d660b7772acef6ce234f536e1f3ec896463af24cfe12b9e430ce"}
Jul 29 16:15:01.697645 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.697550    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:01.699276 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.699245    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt" event={"ID":"405fda58-82b9-4d07-9509-75fff1df213e","Type":"ContainerStarted","Data":"1324dab18fb7de1641445a83f59c4c8bf75cf2e8d4c10bfdc2b9e8b1d9b281e2"}
Jul 29 16:15:01.700791 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.700771    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5" event={"ID":"2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b","Type":"ContainerStarted","Data":"3f68388002783bc100dc794d38a036b3fc9e394b26ed0af62c45aab045b64ee0"}
Jul 29 16:15:01.700895 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.700807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5" event={"ID":"2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b","Type":"ContainerStarted","Data":"c40c3d61510bb911380fb0dcfd6f2233de232a27856e06aabb8d8dc54c871073"}
Jul 29 16:15:01.702279 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.702251    2560 generic.go:356] "Generic (PLEG): container finished" podID="5b81cb20-dcd1-416a-ad56-3963d3dac510" containerID="69d527c12cc2dde6db507adcee0ae652831c8c8ad19acdd826e9b3fecc7983e9" exitCode=0
Jul 29 16:15:01.702345 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.702324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw" event={"ID":"5b81cb20-dcd1-416a-ad56-3963d3dac510","Type":"ContainerDied","Data":"69d527c12cc2dde6db507adcee0ae652831c8c8ad19acdd826e9b3fecc7983e9"}
Jul 29 16:15:01.703672 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.703654    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw" event={"ID":"bbdf705a-ec3f-4e80-8d06-32767ef46863","Type":"ContainerStarted","Data":"db434769122524f3bcef42e06c90959d91be3706c0faa22f9dbf7df5ddea06e4"}
Jul 29 16:15:01.703762 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.703686    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw" event={"ID":"bbdf705a-ec3f-4e80-8d06-32767ef46863","Type":"ContainerStarted","Data":"e777a19c34daca4d45ce0e704a43cb1865f1716d3a81e64f1c6a81bbcf0d5715"}
Jul 29 16:15:01.728538 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.728475    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" podStartSLOduration=1.728455185 podStartE2EDuration="1.728455185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:01.713257378 +0000 UTC m=+3840.007194179" watchObservedRunningTime="2026-07-29 16:15:01.728455185 +0000 UTC m=+3840.022391984"
Jul 29 16:15:01.747534 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.747473    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5" podStartSLOduration=1.7474532219999999 podStartE2EDuration="1.747453222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:01.745718962 +0000 UTC m=+3840.039655797" watchObservedRunningTime="2026-07-29 16:15:01.747453222 +0000 UTC m=+3840.041390026"
Jul 29 16:15:01.763859 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.763805    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt" podStartSLOduration=1.763790594 podStartE2EDuration="1.763790594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:01.760945887 +0000 UTC m=+3840.054882711" watchObservedRunningTime="2026-07-29 16:15:01.763790594 +0000 UTC m=+3840.057727396"
Jul 29 16:15:01.784274 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.784213    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4" podStartSLOduration=1.7841965850000001 podStartE2EDuration="1.784196585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:01.781344778 +0000 UTC m=+3840.075281580" watchObservedRunningTime="2026-07-29 16:15:01.784196585 +0000 UTC m=+3840.078133378"
Jul 29 16:15:01.798157 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:01.798099    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw" podStartSLOduration=1.798082682 podStartE2EDuration="1.798082682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:01.795466939 +0000 UTC m=+3840.089403764" watchObservedRunningTime="2026-07-29 16:15:01.798082682 +0000 UTC m=+3840.092019546"
Jul 29 16:15:02.713785 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.713698    2560 generic.go:356] "Generic (PLEG): container finished" podID="bbdf705a-ec3f-4e80-8d06-32767ef46863" containerID="db434769122524f3bcef42e06c90959d91be3706c0faa22f9dbf7df5ddea06e4" exitCode=0
Jul 29 16:15:02.714302 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.713779    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw" event={"ID":"bbdf705a-ec3f-4e80-8d06-32767ef46863","Type":"ContainerDied","Data":"db434769122524f3bcef42e06c90959d91be3706c0faa22f9dbf7df5ddea06e4"}
Jul 29 16:15:02.715460 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.715434    2560 generic.go:356] "Generic (PLEG): container finished" podID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerID="4552d314f05cb7506725c50f9b47f7b0e6dcd2b93183cd3d2b20289a2784a5c0" exitCode=7
Jul 29 16:15:02.715570 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.715518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" event={"ID":"54c0d42a-5e11-4f8f-b12e-29d59572a561","Type":"ContainerDied","Data":"4552d314f05cb7506725c50f9b47f7b0e6dcd2b93183cd3d2b20289a2784a5c0"}
Jul 29 16:15:02.715667 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.715648    2560 scope.go:117] "RemoveContainer" containerID="4552d314f05cb7506725c50f9b47f7b0e6dcd2b93183cd3d2b20289a2784a5c0"
Jul 29 16:15:02.717268 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.717243    2560 generic.go:356] "Generic (PLEG): container finished" podID="405fda58-82b9-4d07-9509-75fff1df213e" containerID="1324dab18fb7de1641445a83f59c4c8bf75cf2e8d4c10bfdc2b9e8b1d9b281e2" exitCode=0
Jul 29 16:15:02.717384 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.717322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt" event={"ID":"405fda58-82b9-4d07-9509-75fff1df213e","Type":"ContainerDied","Data":"1324dab18fb7de1641445a83f59c4c8bf75cf2e8d4c10bfdc2b9e8b1d9b281e2"}
Jul 29 16:15:02.718887 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.718868    2560 generic.go:356] "Generic (PLEG): container finished" podID="2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b" containerID="3f68388002783bc100dc794d38a036b3fc9e394b26ed0af62c45aab045b64ee0" exitCode=0
Jul 29 16:15:02.718970 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.718910    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5" event={"ID":"2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b","Type":"ContainerDied","Data":"3f68388002783bc100dc794d38a036b3fc9e394b26ed0af62c45aab045b64ee0"}
Jul 29 16:15:02.865958 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:02.865932    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"
Jul 29 16:15:03.727014 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.726918    2560 generic.go:356] "Generic (PLEG): container finished" podID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerID="f75263268d9006f51f8a3e21e9820cc4203f54ed5a2a86e309f6153646848f88" exitCode=7
Jul 29 16:15:03.727507 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.727010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" event={"ID":"54c0d42a-5e11-4f8f-b12e-29d59572a561","Type":"ContainerDied","Data":"f75263268d9006f51f8a3e21e9820cc4203f54ed5a2a86e309f6153646848f88"}
Jul 29 16:15:03.727507 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.727085    2560 scope.go:117] "RemoveContainer" containerID="4552d314f05cb7506725c50f9b47f7b0e6dcd2b93183cd3d2b20289a2784a5c0"
Jul 29 16:15:03.727507 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.727170    2560 scope.go:117] "RemoveContainer" containerID="f75263268d9006f51f8a3e21e9820cc4203f54ed5a2a86e309f6153646848f88"
Jul 29 16:15:03.727507 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:15:03.727459    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml_odh-ai-gateway-infra(54c0d42a-5e11-4f8f-b12e-29d59572a561)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561"
Jul 29 16:15:03.728579 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.728561    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"
Jul 29 16:15:03.728683 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.728604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw" event={"ID":"5b81cb20-dcd1-416a-ad56-3963d3dac510","Type":"ContainerDied","Data":"58c4763caf401abfeb5c4f44175d5363cdafc48277f6018096aaa098ffd6bbad"}
Jul 29 16:15:03.728683 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.728644    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58c4763caf401abfeb5c4f44175d5363cdafc48277f6018096aaa098ffd6bbad"
Jul 29 16:15:03.894509 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.894478    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt"
Jul 29 16:15:03.938786 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.938758    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"
Jul 29 16:15:03.941805 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:03.941781    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"
Jul 29 16:15:04.735609 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.735578    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"
Jul 29 16:15:04.736129 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.735619    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw" event={"ID":"bbdf705a-ec3f-4e80-8d06-32767ef46863","Type":"ContainerDied","Data":"e777a19c34daca4d45ce0e704a43cb1865f1716d3a81e64f1c6a81bbcf0d5715"}
Jul 29 16:15:04.736129 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.735671    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e777a19c34daca4d45ce0e704a43cb1865f1716d3a81e64f1c6a81bbcf0d5715"
Jul 29 16:15:04.737662 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.737621    2560 scope.go:117] "RemoveContainer" containerID="f75263268d9006f51f8a3e21e9820cc4203f54ed5a2a86e309f6153646848f88"
Jul 29 16:15:04.737952 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:15:04.737926    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml_odh-ai-gateway-infra(54c0d42a-5e11-4f8f-b12e-29d59572a561)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561"
Jul 29 16:15:04.738934 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.738919    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt"
Jul 29 16:15:04.739042 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.738951    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755695-8vlkt" event={"ID":"405fda58-82b9-4d07-9509-75fff1df213e","Type":"ContainerDied","Data":"99dc743f440736dcafe4c3ddd6433ab78c24bf8faf7d0a44ba2e1a6d0dc2ab31"}
Jul 29 16:15:04.739042 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.738986    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99dc743f440736dcafe4c3ddd6433ab78c24bf8faf7d0a44ba2e1a6d0dc2ab31"
Jul 29 16:15:04.740250 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.740231    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5" event={"ID":"2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b","Type":"ContainerDied","Data":"c40c3d61510bb911380fb0dcfd6f2233de232a27856e06aabb8d8dc54c871073"}
Jul 29 16:15:04.740250 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.740241    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"
Jul 29 16:15:04.740389 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:04.740261    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c40c3d61510bb911380fb0dcfd6f2233de232a27856e06aabb8d8dc54c871073"
Jul 29 16:15:08.895639 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:08.895600    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:15:12.722802 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:12.722766    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:15:18.160970 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:18.160941    2560 scope.go:117] "RemoveContainer" containerID="f75263268d9006f51f8a3e21e9820cc4203f54ed5a2a86e309f6153646848f88"
Jul 29 16:15:18.824114 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:18.824020    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" event={"ID":"54c0d42a-5e11-4f8f-b12e-29d59572a561","Type":"ContainerStarted","Data":"9dcb2b7dd34c75b4a8a7580e8f34eb95100037aa912f8f6704c04a39a63cdc67"}
Jul 29 16:15:19.831843 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:19.831806    2560 generic.go:356] "Generic (PLEG): container finished" podID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerID="9dcb2b7dd34c75b4a8a7580e8f34eb95100037aa912f8f6704c04a39a63cdc67" exitCode=0
Jul 29 16:15:19.832259 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:19.831867    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" event={"ID":"54c0d42a-5e11-4f8f-b12e-29d59572a561","Type":"ContainerDied","Data":"9dcb2b7dd34c75b4a8a7580e8f34eb95100037aa912f8f6704c04a39a63cdc67"}
Jul 29 16:15:19.832259 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:19.831943    2560 scope.go:117] "RemoveContainer" containerID="f75263268d9006f51f8a3e21e9820cc4203f54ed5a2a86e309f6153646848f88"
Jul 29 16:15:19.857708 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:19.857668    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"]
Jul 29 16:15:20.965392 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:20.965365    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"
Jul 29 16:15:21.846791 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:21.846756    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml" event={"ID":"54c0d42a-5e11-4f8f-b12e-29d59572a561","Type":"ContainerDied","Data":"c7d7ef0547e4e3df458610a15d6e3b788a23904f472152900477efca7ab70693"}
Jul 29 16:15:21.847009 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:21.846814    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7d7ef0547e4e3df458610a15d6e3b788a23904f472152900477efca7ab70693"
Jul 29 16:15:21.847009 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:21.846763    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"
Jul 29 16:15:21.878926 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:21.878869    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"]
Jul 29 16:15:21.882920 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:21.882894    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-16a6d7-29755695-6scml"]
Jul 29 16:15:22.166882 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:22.166798    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" path="/var/lib/kubelet/pods/54c0d42a-5e11-4f8f-b12e-29d59572a561/volumes"
Jul 29 16:15:57.436913 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.436829    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"]
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437264    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437276    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437291    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437297    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437304    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437310    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437316    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="405fda58-82b9-4d07-9509-75fff1df213e" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437321    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="405fda58-82b9-4d07-9509-75fff1df213e" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437331    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbdf705a-ec3f-4e80-8d06-32767ef46863" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437336    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbdf705a-ec3f-4e80-8d06-32767ef46863" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437342    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437347    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437355    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b81cb20-dcd1-416a-ad56-3963d3dac510" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437360    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b81cb20-dcd1-416a-ad56-3963d3dac510" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437420    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbdf705a-ec3f-4e80-8d06-32767ef46863" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437429    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437435    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b" containerName="cleanup"
Jul 29 16:15:57.437437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437442    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b81cb20-dcd1-416a-ad56-3963d3dac510" containerName="cleanup"
Jul 29 16:15:57.438020 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437449    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:15:57.438020 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.437456    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="405fda58-82b9-4d07-9509-75fff1df213e" containerName="cleanup"
Jul 29 16:15:57.440738 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.440715    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.443754 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.443719    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6fccfc\"/\"openshift-service-ca.crt\""
Jul 29 16:15:57.443754 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.443725    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6fccfc\"/\"kube-root-ca.crt\""
Jul 29 16:15:57.444511 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.444495    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6fccfc\"/\"ipp-route-6fccfc-kserve-self-signed-certs\""
Jul 29 16:15:57.445566 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.445548    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6fccfc\"/\"default-dockercfg-5npvd\""
Jul 29 16:15:57.452745 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.452718    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"]
Jul 29 16:15:57.530263 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.530210    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-tmp-dir\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.530263 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.530251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49049595-b12b-445c-81ad-de36e02cb68e-tls-certs\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.530545 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.530312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-dshm\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.530545 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.530338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v249r\" (UniqueName: \"kubernetes.io/projected/49049595-b12b-445c-81ad-de36e02cb68e-kube-api-access-v249r\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.530545 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.530420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-home\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.530545 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.530477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-model-cache\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.631576 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.631533    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-model-cache\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.631771 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.631605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-tmp-dir\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.631771 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.631636    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49049595-b12b-445c-81ad-de36e02cb68e-tls-certs\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.631771 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.631680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-dshm\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.631771 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.631710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v249r\" (UniqueName: \"kubernetes.io/projected/49049595-b12b-445c-81ad-de36e02cb68e-kube-api-access-v249r\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.631771 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.631764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-home\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.632072 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.632048    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-model-cache\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.632072 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.632065    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-home\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.632244 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.632104    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-tmp-dir\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.634024 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.633997    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-dshm\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.634328 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.634310    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49049595-b12b-445c-81ad-de36e02cb68e-tls-certs\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.641804 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.641776    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v249r\" (UniqueName: \"kubernetes.io/projected/49049595-b12b-445c-81ad-de36e02cb68e-kube-api-access-v249r\") pod \"ipp-route-6fccfc-kserve-6c8bcc495f-x98wg\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") " pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.752854 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.752759    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:15:57.893412 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:57.893387    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"]
Jul 29 16:15:57.895218 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:15:57.895164    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49049595_b12b_445c_81ad_de36e02cb68e.slice/crio-9414ce99d91313800df1f5116c47e420cd135a6bb9029c9f5f148e7b4be0d303 WatchSource:0}: Error finding container 9414ce99d91313800df1f5116c47e420cd135a6bb9029c9f5f148e7b4be0d303: Status 404 returned error can't find the container with id 9414ce99d91313800df1f5116c47e420cd135a6bb9029c9f5f148e7b4be0d303
Jul 29 16:15:58.057592 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:58.057490    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg" event={"ID":"49049595-b12b-445c-81ad-de36e02cb68e","Type":"ContainerStarted","Data":"668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22"}
Jul 29 16:15:58.057592 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:58.057538    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg" event={"ID":"49049595-b12b-445c-81ad-de36e02cb68e","Type":"ContainerStarted","Data":"9414ce99d91313800df1f5116c47e420cd135a6bb9029c9f5f148e7b4be0d303"}
Jul 29 16:15:58.077615 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:15:58.077560    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg" podStartSLOduration=1.077541947 podStartE2EDuration="1.077541947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:58.077330244 +0000 UTC m=+3896.371267048" watchObservedRunningTime="2026-07-29 16:15:58.077541947 +0000 UTC m=+3896.371478749"
Jul 29 16:16:01.395414 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:01.395372    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:16:02.495753 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:02.495722    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:16:02.504716 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:02.504689    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:16:02.517011 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:02.516980    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:16:02.525699 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:02.525674    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sscmw_c6e6ae73-f87c-4666-9d7b-28584bb70925/ovn-acl-logging/0.log"
Jul 29 16:16:07.753059 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:07.753019    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:16:07.753656 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:07.753074    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:16:07.765930 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:07.765902    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:16:08.127818 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:08.127786    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:16:28.586825 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:28.586787    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:16:39.275603 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:39.275568    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v_bf87d4a5-1bc4-49ad-aaff-a3625cc879a0/payload-processing/0.log"
Jul 29 16:16:44.211283 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.211245    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"]
Jul 29 16:16:44.211725 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.211712    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c0d42a-5e11-4f8f-b12e-29d59572a561" containerName="cleanup"
Jul 29 16:16:44.215029 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.215010    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.217762 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.217725    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-86568b\""
Jul 29 16:16:44.227270 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.227242    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"]
Jul 29 16:16:44.270792 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.270753    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"]
Jul 29 16:16:44.274434 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.274414    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.276700 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.276680    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-86568b-dockercfg-62mnb\""
Jul 29 16:16:44.284919 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.284889    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"]
Jul 29 16:16:44.337888 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.337855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004448e4-51dc-4241-978d-aaa67246ccb4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.338083 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.337920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4f2k\" (UniqueName: \"kubernetes.io/projected/004448e4-51dc-4241-978d-aaa67246ccb4-kube-api-access-t4f2k\") pod \"payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.338083 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.337952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql92z\" (UniqueName: \"kubernetes.io/projected/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-kube-api-access-ql92z\") pod \"payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.338083 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.337985    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.438455 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.438418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.438656 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.438487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004448e4-51dc-4241-978d-aaa67246ccb4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.438656 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.438549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4f2k\" (UniqueName: \"kubernetes.io/projected/004448e4-51dc-4241-978d-aaa67246ccb4-kube-api-access-t4f2k\") pod \"payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.438656 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.438575    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql92z\" (UniqueName: \"kubernetes.io/projected/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-kube-api-access-ql92z\") pod \"payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.439329 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.439306    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004448e4-51dc-4241-978d-aaa67246ccb4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.439396 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.439332    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.454959 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.454927    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql92z\" (UniqueName: \"kubernetes.io/projected/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-kube-api-access-ql92z\") pod \"payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.455104 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.455039    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4f2k\" (UniqueName: \"kubernetes.io/projected/004448e4-51dc-4241-978d-aaa67246ccb4-kube-api-access-t4f2k\") pod \"payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.527585 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.527491    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:44.585609 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.585577    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:44.680261 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.680158    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"]
Jul 29 16:16:44.683475 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:16:44.683431    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod004448e4_51dc_4241_978d_aaa67246ccb4.slice/crio-26b7486211734b9b8016f930e3527b199362442dd53245dc85589d95105a8bdf WatchSource:0}: Error finding container 26b7486211734b9b8016f930e3527b199362442dd53245dc85589d95105a8bdf: Status 404 returned error can't find the container with id 26b7486211734b9b8016f930e3527b199362442dd53245dc85589d95105a8bdf
Jul 29 16:16:44.745485 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:44.744071    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"]
Jul 29 16:16:44.748068 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:16:44.747887    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4baba4_a4f3_4c8d_a1f0_7e8c9c25034d.slice/crio-292e424553ca292cbd0288573f1f6646c9e1f7f635284f8472ee319eb107b030 WatchSource:0}: Error finding container 292e424553ca292cbd0288573f1f6646c9e1f7f635284f8472ee319eb107b030: Status 404 returned error can't find the container with id 292e424553ca292cbd0288573f1f6646c9e1f7f635284f8472ee319eb107b030
Jul 29 16:16:45.334286 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.334248    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9" event={"ID":"004448e4-51dc-4241-978d-aaa67246ccb4","Type":"ContainerStarted","Data":"c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e"}
Jul 29 16:16:45.334789 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.334314    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9" event={"ID":"004448e4-51dc-4241-978d-aaa67246ccb4","Type":"ContainerStarted","Data":"26b7486211734b9b8016f930e3527b199362442dd53245dc85589d95105a8bdf"}
Jul 29 16:16:45.334789 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.334428    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:16:45.335652 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.335622    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg" event={"ID":"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d","Type":"ContainerStarted","Data":"70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99"}
Jul 29 16:16:45.335780 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.335675    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg" event={"ID":"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d","Type":"ContainerStarted","Data":"292e424553ca292cbd0288573f1f6646c9e1f7f635284f8472ee319eb107b030"}
Jul 29 16:16:45.335780 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.335715    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:45.354142 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.354092    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9" podStartSLOduration=1.354078392 podStartE2EDuration="1.354078392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:16:45.35163212 +0000 UTC m=+3943.645568938" watchObservedRunningTime="2026-07-29 16:16:45.354078392 +0000 UTC m=+3943.648015193"
Jul 29 16:16:45.368539 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:45.368489    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg" podStartSLOduration=1.368478146 podStartE2EDuration="1.368478146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:16:45.367368998 +0000 UTC m=+3943.661305802" watchObservedRunningTime="2026-07-29 16:16:45.368478146 +0000 UTC m=+3943.662414979"
Jul 29 16:16:46.478327 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:46.478291    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:16:56.345366 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:56.345277    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:16:56.345366 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:16:56.345365    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:17:02.783026 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:02.782942    2560 scope.go:117] "RemoveContainer" containerID="fed7218798f74a3e5bbb2a88d8a56d2ac1579f7ab20872e396f6d7526c2dd2fd"
Jul 29 16:17:14.120590 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.120553    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"]
Jul 29 16:17:14.121129 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.120784    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9" podUID="004448e4-51dc-4241-978d-aaa67246ccb4" containerName="payload-processing" containerID="cri-o://c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e" gracePeriod=30
Jul 29 16:17:14.131033 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.130921    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"]
Jul 29 16:17:14.131233 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.131161    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg" podUID="1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" containerName="payload-pre-processing" containerID="cri-o://70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99" gracePeriod=30
Jul 29 16:17:14.381710 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.381644    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:17:14.410932 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.410903    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:17:14.428927 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.428895    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql92z\" (UniqueName: \"kubernetes.io/projected/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-kube-api-access-ql92z\") pod \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") "
Jul 29 16:17:14.429524 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.428938    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-plugins-config-volume\") pod \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\" (UID: \"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d\") "
Jul 29 16:17:14.429524 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.429015    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4f2k\" (UniqueName: \"kubernetes.io/projected/004448e4-51dc-4241-978d-aaa67246ccb4-kube-api-access-t4f2k\") pod \"004448e4-51dc-4241-978d-aaa67246ccb4\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") "
Jul 29 16:17:14.429524 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.429055    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004448e4-51dc-4241-978d-aaa67246ccb4-plugins-config-volume\") pod \"004448e4-51dc-4241-978d-aaa67246ccb4\" (UID: \"004448e4-51dc-4241-978d-aaa67246ccb4\") "
Jul 29 16:17:14.429842 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.429772    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/004448e4-51dc-4241-978d-aaa67246ccb4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "004448e4-51dc-4241-978d-aaa67246ccb4" (UID: "004448e4-51dc-4241-978d-aaa67246ccb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:14.430306 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.430274    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" (UID: "1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:14.432006 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.431981    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-kube-api-access-ql92z" (OuterVolumeSpecName: "kube-api-access-ql92z") pod "1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" (UID: "1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d"). InnerVolumeSpecName "kube-api-access-ql92z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:17:14.432652 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.432620    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/004448e4-51dc-4241-978d-aaa67246ccb4-kube-api-access-t4f2k" (OuterVolumeSpecName: "kube-api-access-t4f2k") pod "004448e4-51dc-4241-978d-aaa67246ccb4" (UID: "004448e4-51dc-4241-978d-aaa67246ccb4"). InnerVolumeSpecName "kube-api-access-t4f2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:17:14.518221 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.518158    2560 generic.go:356] "Generic (PLEG): container finished" podID="004448e4-51dc-4241-978d-aaa67246ccb4" containerID="c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e" exitCode=0
Jul 29 16:17:14.518221 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.518214    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9" event={"ID":"004448e4-51dc-4241-978d-aaa67246ccb4","Type":"ContainerDied","Data":"c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e"}
Jul 29 16:17:14.518448 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.518230    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"
Jul 29 16:17:14.518448 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.518276    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9" event={"ID":"004448e4-51dc-4241-978d-aaa67246ccb4","Type":"ContainerDied","Data":"26b7486211734b9b8016f930e3527b199362442dd53245dc85589d95105a8bdf"}
Jul 29 16:17:14.518448 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.518305    2560 scope.go:117] "RemoveContainer" containerID="c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e"
Jul 29 16:17:14.519875 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.519853    2560 generic.go:356] "Generic (PLEG): container finished" podID="1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" containerID="70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99" exitCode=0
Jul 29 16:17:14.519989 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.519910    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"
Jul 29 16:17:14.519989 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.519943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg" event={"ID":"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d","Type":"ContainerDied","Data":"70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99"}
Jul 29 16:17:14.520087 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.519995    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg" event={"ID":"1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d","Type":"ContainerDied","Data":"292e424553ca292cbd0288573f1f6646c9e1f7f635284f8472ee319eb107b030"}
Jul 29 16:17:14.528944 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.528925    2560 scope.go:117] "RemoveContainer" containerID="c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e"
Jul 29 16:17:14.529355 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:17:14.529233    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e\": container with ID starting with c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e not found: ID does not exist" containerID="c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e"
Jul 29 16:17:14.529355 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.529257    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e"} err="failed to get container status \"c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e\": rpc error: code = NotFound desc = could not find container \"c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e\": container with ID starting with c3a4dd7de09837979c16e935facd5cafdc4fa00b55e04883acb3e2c8225dc22e not found: ID does not exist"
Jul 29 16:17:14.529355 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.529272    2560 scope.go:117] "RemoveContainer" containerID="70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99"
Jul 29 16:17:14.530079 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.530059    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql92z\" (UniqueName: \"kubernetes.io/projected/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-kube-api-access-ql92z\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:17:14.530170 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.530084    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:17:14.530170 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.530098    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4f2k\" (UniqueName: \"kubernetes.io/projected/004448e4-51dc-4241-978d-aaa67246ccb4-kube-api-access-t4f2k\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:17:14.530170 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.530111    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004448e4-51dc-4241-978d-aaa67246ccb4-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:17:14.538209 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.538177    2560 scope.go:117] "RemoveContainer" containerID="70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99"
Jul 29 16:17:14.538470 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:17:14.538451    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99\": container with ID starting with 70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99 not found: ID does not exist" containerID="70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99"
Jul 29 16:17:14.538537 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.538482    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99"} err="failed to get container status \"70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99\": rpc error: code = NotFound desc = could not find container \"70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99\": container with ID starting with 70172e6ca92e8a6e9ea563b9d8474f709bb184c4b04649c509fb2eba78bc2b99 not found: ID does not exist"
Jul 29 16:17:14.548158 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.548102    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"]
Jul 29 16:17:14.550863 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.550837    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-86568b-7cc9c866-txcs9"]
Jul 29 16:17:14.563691 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.563668    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"]
Jul 29 16:17:14.571071 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:14.570273    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-86568b-7b7cdbf956-bt6sg"]
Jul 29 16:17:16.166660 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:16.166626    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="004448e4-51dc-4241-978d-aaa67246ccb4" path="/var/lib/kubelet/pods/004448e4-51dc-4241-978d-aaa67246ccb4/volumes"
Jul 29 16:17:16.167066 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:16.166983    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" path="/var/lib/kubelet/pods/1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d/volumes"
Jul 29 16:17:18.593679 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:18.593635    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:17:24.876019 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.875980    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"]
Jul 29 16:17:24.876568 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.876393    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" containerName="payload-pre-processing"
Jul 29 16:17:24.876568 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.876406    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" containerName="payload-pre-processing"
Jul 29 16:17:24.876568 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.876437    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="004448e4-51dc-4241-978d-aaa67246ccb4" containerName="payload-processing"
Jul 29 16:17:24.876568 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.876443    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="004448e4-51dc-4241-978d-aaa67246ccb4" containerName="payload-processing"
Jul 29 16:17:24.876568 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.876499    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f4baba4-a4f3-4c8d-a1f0-7e8c9c25034d" containerName="payload-pre-processing"
Jul 29 16:17:24.876568 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.876511    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="004448e4-51dc-4241-978d-aaa67246ccb4" containerName="payload-processing"
Jul 29 16:17:24.879381 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.879360    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"
Jul 29 16:17:24.881983 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.881960    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:17:24.882110 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.881959    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 16:17:24.888708 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.888686    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"]
Jul 29 16:17:24.923842 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:24.923806    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3492e111-970b-4d3a-b925-e1db0bbd11e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9\" (UID: \"3492e111-970b-4d3a-b925-e1db0bbd11e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"
Jul 29 16:17:25.024492 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:25.024451    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3492e111-970b-4d3a-b925-e1db0bbd11e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9\" (UID: \"3492e111-970b-4d3a-b925-e1db0bbd11e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"
Jul 29 16:17:25.025037 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:25.025017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3492e111-970b-4d3a-b925-e1db0bbd11e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9\" (UID: \"3492e111-970b-4d3a-b925-e1db0bbd11e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"
Jul 29 16:17:25.191009 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:25.190906    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"
Jul 29 16:17:25.333460 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:25.333433    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"]
Jul 29 16:17:25.334371 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:17:25.334345    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3492e111_970b_4d3a_b925_e1db0bbd11e0.slice/crio-654daebb4b25aa022c1f90388a9da43ba53a9940fb063e161b0df2641bf7e4df WatchSource:0}: Error finding container 654daebb4b25aa022c1f90388a9da43ba53a9940fb063e161b0df2641bf7e4df: Status 404 returned error can't find the container with id 654daebb4b25aa022c1f90388a9da43ba53a9940fb063e161b0df2641bf7e4df
Jul 29 16:17:25.586530 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:25.586494    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9" event={"ID":"3492e111-970b-4d3a-b925-e1db0bbd11e0","Type":"ContainerStarted","Data":"654daebb4b25aa022c1f90388a9da43ba53a9940fb063e161b0df2641bf7e4df"}
Jul 29 16:17:26.593961 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:26.593925    2560 generic.go:356] "Generic (PLEG): container finished" podID="3492e111-970b-4d3a-b925-e1db0bbd11e0" containerID="1e8bb534d5777ccf3eeec840574cc8c317d06af7d608d2669bb7b9de600b0a69" exitCode=0
Jul 29 16:17:26.594430 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:26.594014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9" event={"ID":"3492e111-970b-4d3a-b925-e1db0bbd11e0","Type":"ContainerDied","Data":"1e8bb534d5777ccf3eeec840574cc8c317d06af7d608d2669bb7b9de600b0a69"}
Jul 29 16:17:27.742328 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:27.742299    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"
Jul 29 16:17:27.848266 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:27.848232    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3492e111-970b-4d3a-b925-e1db0bbd11e0-maas-api-service-ca\") pod \"3492e111-970b-4d3a-b925-e1db0bbd11e0\" (UID: \"3492e111-970b-4d3a-b925-e1db0bbd11e0\") "
Jul 29 16:17:27.848642 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:27.848614    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3492e111-970b-4d3a-b925-e1db0bbd11e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3492e111-970b-4d3a-b925-e1db0bbd11e0" (UID: "3492e111-970b-4d3a-b925-e1db0bbd11e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:27.949050 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:27.948966    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3492e111-970b-4d3a-b925-e1db0bbd11e0-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:17:28.606787 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:28.606752    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9" event={"ID":"3492e111-970b-4d3a-b925-e1db0bbd11e0","Type":"ContainerDied","Data":"654daebb4b25aa022c1f90388a9da43ba53a9940fb063e161b0df2641bf7e4df"}
Jul 29 16:17:28.606981 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:28.606802    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="654daebb4b25aa022c1f90388a9da43ba53a9940fb063e161b0df2641bf7e4df"
Jul 29 16:17:28.606981 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:28.606769    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"
Jul 29 16:17:39.982620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:39.982576    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:17:40.413622 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.413584    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"]
Jul 29 16:17:40.417212 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.417174    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fccfc-29755695-nsbgw"]
Jul 29 16:17:40.448318 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.448285    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"]
Jul 29 16:17:40.448552 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.448525    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v" podUID="bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" containerName="payload-processing" containerID="cri-o://a7d80e4de8e0084b6d39f74abf00ddcff1d9487f47cdad063e3c1fca7e18dc64" gracePeriod=30
Jul 29 16:17:40.621050 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.620784    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"]
Jul 29 16:17:40.625273 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.625239    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6fccfc-nvjj9"]
Jul 29 16:17:40.677011 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.676927    2560 generic.go:356] "Generic (PLEG): container finished" podID="bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" containerID="a7d80e4de8e0084b6d39f74abf00ddcff1d9487f47cdad063e3c1fca7e18dc64" exitCode=0
Jul 29 16:17:40.677011 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.677000    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v" event={"ID":"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0","Type":"ContainerDied","Data":"a7d80e4de8e0084b6d39f74abf00ddcff1d9487f47cdad063e3c1fca7e18dc64"}
Jul 29 16:17:40.713260 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.713231    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:17:40.859859 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.859801    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6z7q\" (UniqueName: \"kubernetes.io/projected/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-kube-api-access-r6z7q\") pod \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") "
Jul 29 16:17:40.859859 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.859861    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-plugins-config-volume\") pod \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\" (UID: \"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0\") "
Jul 29 16:17:40.860277 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.860252    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" (UID: "bf87d4a5-1bc4-49ad-aaff-a3625cc879a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:40.862094 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.862066    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-kube-api-access-r6z7q" (OuterVolumeSpecName: "kube-api-access-r6z7q") pod "bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" (UID: "bf87d4a5-1bc4-49ad-aaff-a3625cc879a0"). InnerVolumeSpecName "kube-api-access-r6z7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:17:40.961003 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.960899    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6z7q\" (UniqueName: \"kubernetes.io/projected/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-kube-api-access-r6z7q\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:17:40.961003 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:40.960942    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:17:41.683763 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:41.683731    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"
Jul 29 16:17:41.684248 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:41.683762    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v" event={"ID":"bf87d4a5-1bc4-49ad-aaff-a3625cc879a0","Type":"ContainerDied","Data":"a119fb74514a3cecb71741f7db827d3af8138882905a4ea00e6a95657d735cb3"}
Jul 29 16:17:41.684248 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:41.683825    2560 scope.go:117] "RemoveContainer" containerID="a7d80e4de8e0084b6d39f74abf00ddcff1d9487f47cdad063e3c1fca7e18dc64"
Jul 29 16:17:41.708991 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:41.708961    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"]
Jul 29 16:17:41.711364 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:41.711338    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6fccfc-67d787c89b-xzq2v"]
Jul 29 16:17:42.167358 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:42.167324    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3492e111-970b-4d3a-b925-e1db0bbd11e0" path="/var/lib/kubelet/pods/3492e111-970b-4d3a-b925-e1db0bbd11e0/volumes"
Jul 29 16:17:42.167736 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:42.167713    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbdf705a-ec3f-4e80-8d06-32767ef46863" path="/var/lib/kubelet/pods/bbdf705a-ec3f-4e80-8d06-32767ef46863/volumes"
Jul 29 16:17:42.168063 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:42.168046    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" path="/var/lib/kubelet/pods/bf87d4a5-1bc4-49ad-aaff-a3625cc879a0/volumes"
Jul 29 16:17:46.009685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:46.009645    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"]
Jul 29 16:17:50.791051 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:17:50.791005    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:18:01.029136 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.029064    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg" podUID="49049595-b12b-445c-81ad-de36e02cb68e" containerName="main" containerID="cri-o://668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22" gracePeriod=45
Jul 29 16:18:01.286331 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.286260    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:18:01.347480 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347439    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49049595-b12b-445c-81ad-de36e02cb68e-tls-certs\") pod \"49049595-b12b-445c-81ad-de36e02cb68e\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") "
Jul 29 16:18:01.347681 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347504    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-dshm\") pod \"49049595-b12b-445c-81ad-de36e02cb68e\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") "
Jul 29 16:18:01.347681 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347561    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-tmp-dir\") pod \"49049595-b12b-445c-81ad-de36e02cb68e\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") "
Jul 29 16:18:01.347681 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347587    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v249r\" (UniqueName: \"kubernetes.io/projected/49049595-b12b-445c-81ad-de36e02cb68e-kube-api-access-v249r\") pod \"49049595-b12b-445c-81ad-de36e02cb68e\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") "
Jul 29 16:18:01.347681 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347618    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-model-cache\") pod \"49049595-b12b-445c-81ad-de36e02cb68e\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") "
Jul 29 16:18:01.347681 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347647    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-home\") pod \"49049595-b12b-445c-81ad-de36e02cb68e\" (UID: \"49049595-b12b-445c-81ad-de36e02cb68e\") "
Jul 29 16:18:01.347973 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347879    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "49049595-b12b-445c-81ad-de36e02cb68e" (UID: "49049595-b12b-445c-81ad-de36e02cb68e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:18:01.348035 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.347901    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-model-cache" (OuterVolumeSpecName: "model-cache") pod "49049595-b12b-445c-81ad-de36e02cb68e" (UID: "49049595-b12b-445c-81ad-de36e02cb68e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:18:01.348121 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.348091    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-home" (OuterVolumeSpecName: "home") pod "49049595-b12b-445c-81ad-de36e02cb68e" (UID: "49049595-b12b-445c-81ad-de36e02cb68e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:18:01.349719 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.349696    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-dshm" (OuterVolumeSpecName: "dshm") pod "49049595-b12b-445c-81ad-de36e02cb68e" (UID: "49049595-b12b-445c-81ad-de36e02cb68e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 16:18:01.350231 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.350206    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49049595-b12b-445c-81ad-de36e02cb68e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "49049595-b12b-445c-81ad-de36e02cb68e" (UID: "49049595-b12b-445c-81ad-de36e02cb68e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:18:01.350284 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.350224    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49049595-b12b-445c-81ad-de36e02cb68e-kube-api-access-v249r" (OuterVolumeSpecName: "kube-api-access-v249r") pod "49049595-b12b-445c-81ad-de36e02cb68e" (UID: "49049595-b12b-445c-81ad-de36e02cb68e"). InnerVolumeSpecName "kube-api-access-v249r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:01.449055 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.449001    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49049595-b12b-445c-81ad-de36e02cb68e-tls-certs\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:01.449055 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.449047    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-dshm\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:01.449055 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.449060    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-tmp-dir\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:01.449346 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.449072    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v249r\" (UniqueName: \"kubernetes.io/projected/49049595-b12b-445c-81ad-de36e02cb68e-kube-api-access-v249r\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:01.449346 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.449084    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-model-cache\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:01.449346 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.449095    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49049595-b12b-445c-81ad-de36e02cb68e-home\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:01.805883 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.805846    2560 generic.go:356] "Generic (PLEG): container finished" podID="49049595-b12b-445c-81ad-de36e02cb68e" containerID="668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22" exitCode=0
Jul 29 16:18:01.806108 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.805894    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"
Jul 29 16:18:01.806108 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.805905    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg" event={"ID":"49049595-b12b-445c-81ad-de36e02cb68e","Type":"ContainerDied","Data":"668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22"}
Jul 29 16:18:01.806108 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.805943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg" event={"ID":"49049595-b12b-445c-81ad-de36e02cb68e","Type":"ContainerDied","Data":"9414ce99d91313800df1f5116c47e420cd135a6bb9029c9f5f148e7b4be0d303"}
Jul 29 16:18:01.806108 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.805965    2560 scope.go:117] "RemoveContainer" containerID="668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22"
Jul 29 16:18:01.816172 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.816144    2560 scope.go:117] "RemoveContainer" containerID="668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22"
Jul 29 16:18:01.816467 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:18:01.816447    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22\": container with ID starting with 668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22 not found: ID does not exist" containerID="668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22"
Jul 29 16:18:01.816519 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.816478    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22"} err="failed to get container status \"668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22\": rpc error: code = NotFound desc = could not find container \"668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22\": container with ID starting with 668cd3c6673bd27c0a14f87a8dce42da62d37e8779f558601bc9a7a46bfb7f22 not found: ID does not exist"
Jul 29 16:18:01.833839 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.833807    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"]
Jul 29 16:18:01.842198 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:01.842151    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-6fccfc/ipp-route-6fccfc-kserve-6c8bcc495f-x98wg"]
Jul 29 16:18:02.167427 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:02.167396    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49049595-b12b-445c-81ad-de36e02cb68e" path="/var/lib/kubelet/pods/49049595-b12b-445c-81ad-de36e02cb68e/volumes"
Jul 29 16:18:30.143967 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.143877    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"]
Jul 29 16:18:30.144437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144330    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49049595-b12b-445c-81ad-de36e02cb68e" containerName="main"
Jul 29 16:18:30.144437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144345    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="49049595-b12b-445c-81ad-de36e02cb68e" containerName="main"
Jul 29 16:18:30.144437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144357    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3492e111-970b-4d3a-b925-e1db0bbd11e0" containerName="revoke-keys"
Jul 29 16:18:30.144437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144363    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3492e111-970b-4d3a-b925-e1db0bbd11e0" containerName="revoke-keys"
Jul 29 16:18:30.144437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144383    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" containerName="payload-processing"
Jul 29 16:18:30.144437 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144389    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" containerName="payload-processing"
Jul 29 16:18:30.144635 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144455    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3492e111-970b-4d3a-b925-e1db0bbd11e0" containerName="revoke-keys"
Jul 29 16:18:30.144635 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144464    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf87d4a5-1bc4-49ad-aaff-a3625cc879a0" containerName="payload-processing"
Jul 29 16:18:30.144635 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.144470    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="49049595-b12b-445c-81ad-de36e02cb68e" containerName="main"
Jul 29 16:18:30.147567 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.147548    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"
Jul 29 16:18:30.150237 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.150210    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 16:18:30.150427 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.150246    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:18:30.154754 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.154725    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"]
Jul 29 16:18:30.209515 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.209474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cceb4693-fe21-4ecd-93cb-a37021129f2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr\" (UID: \"cceb4693-fe21-4ecd-93cb-a37021129f2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"
Jul 29 16:18:30.310259 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.310214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cceb4693-fe21-4ecd-93cb-a37021129f2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr\" (UID: \"cceb4693-fe21-4ecd-93cb-a37021129f2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"
Jul 29 16:18:30.310886 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.310861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cceb4693-fe21-4ecd-93cb-a37021129f2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr\" (UID: \"cceb4693-fe21-4ecd-93cb-a37021129f2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"
Jul 29 16:18:30.459484 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.459392    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"
Jul 29 16:18:30.597487 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.597461    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"]
Jul 29 16:18:30.600230 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:18:30.600198    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcceb4693_fe21_4ecd_93cb_a37021129f2a.slice/crio-e706c8589b682c5caaacc36c085c5c20d9472b563d0d2e20de36f939ef105394 WatchSource:0}: Error finding container e706c8589b682c5caaacc36c085c5c20d9472b563d0d2e20de36f939ef105394: Status 404 returned error can't find the container with id e706c8589b682c5caaacc36c085c5c20d9472b563d0d2e20de36f939ef105394
Jul 29 16:18:30.974513 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.974471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr" event={"ID":"cceb4693-fe21-4ecd-93cb-a37021129f2a","Type":"ContainerStarted","Data":"43825d11a811e957b3657becc2ae8a0bd4d2516281b17b9160c003165ce7726f"}
Jul 29 16:18:30.974707 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.974557    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr" event={"ID":"cceb4693-fe21-4ecd-93cb-a37021129f2a","Type":"ContainerStarted","Data":"e706c8589b682c5caaacc36c085c5c20d9472b563d0d2e20de36f939ef105394"}
Jul 29 16:18:30.991962 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:30.991903    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr" podStartSLOduration=0.991887071 podStartE2EDuration="991.887071ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:18:30.990201238 +0000 UTC m=+4049.284138051" watchObservedRunningTime="2026-07-29 16:18:30.991887071 +0000 UTC m=+4049.285823872"
Jul 29 16:18:31.981713 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:31.981678    2560 generic.go:356] "Generic (PLEG): container finished" podID="cceb4693-fe21-4ecd-93cb-a37021129f2a" containerID="43825d11a811e957b3657becc2ae8a0bd4d2516281b17b9160c003165ce7726f" exitCode=0
Jul 29 16:18:31.982243 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:31.981738    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr" event={"ID":"cceb4693-fe21-4ecd-93cb-a37021129f2a","Type":"ContainerDied","Data":"43825d11a811e957b3657becc2ae8a0bd4d2516281b17b9160c003165ce7726f"}
Jul 29 16:18:33.125394 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:33.125365    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"
Jul 29 16:18:33.239459 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:33.239375    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cceb4693-fe21-4ecd-93cb-a37021129f2a-maas-api-service-ca\") pod \"cceb4693-fe21-4ecd-93cb-a37021129f2a\" (UID: \"cceb4693-fe21-4ecd-93cb-a37021129f2a\") "
Jul 29 16:18:33.239759 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:33.239736    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cceb4693-fe21-4ecd-93cb-a37021129f2a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cceb4693-fe21-4ecd-93cb-a37021129f2a" (UID: "cceb4693-fe21-4ecd-93cb-a37021129f2a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:33.340398 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:33.340360    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cceb4693-fe21-4ecd-93cb-a37021129f2a-maas-api-service-ca\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:33.994685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:33.994649    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr" event={"ID":"cceb4693-fe21-4ecd-93cb-a37021129f2a","Type":"ContainerDied","Data":"e706c8589b682c5caaacc36c085c5c20d9472b563d0d2e20de36f939ef105394"}
Jul 29 16:18:33.994685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:33.994668    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"
Jul 29 16:18:33.994685 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:33.994699    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e706c8589b682c5caaacc36c085c5c20d9472b563d0d2e20de36f939ef105394"
Jul 29 16:18:35.091762 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.091727    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"]
Jul 29 16:18:35.092270 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.091956    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4" podUID="458c6823-2a62-4357-8f2d-b798e43529ad" containerName="payload-pre-processing" containerID="cri-o://6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b" gracePeriod=30
Jul 29 16:18:35.266995 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.266958    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"]
Jul 29 16:18:35.270877 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.270845    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-16a6d7-68dnr"]
Jul 29 16:18:35.355970 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.355947    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:18:35.460259 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.460219    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458c6823-2a62-4357-8f2d-b798e43529ad-plugins-config-volume\") pod \"458c6823-2a62-4357-8f2d-b798e43529ad\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") "
Jul 29 16:18:35.460460 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.460284    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqjlt\" (UniqueName: \"kubernetes.io/projected/458c6823-2a62-4357-8f2d-b798e43529ad-kube-api-access-fqjlt\") pod \"458c6823-2a62-4357-8f2d-b798e43529ad\" (UID: \"458c6823-2a62-4357-8f2d-b798e43529ad\") "
Jul 29 16:18:35.460620 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.460597    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458c6823-2a62-4357-8f2d-b798e43529ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "458c6823-2a62-4357-8f2d-b798e43529ad" (UID: "458c6823-2a62-4357-8f2d-b798e43529ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:35.462407 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.462373    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/458c6823-2a62-4357-8f2d-b798e43529ad-kube-api-access-fqjlt" (OuterVolumeSpecName: "kube-api-access-fqjlt") pod "458c6823-2a62-4357-8f2d-b798e43529ad" (UID: "458c6823-2a62-4357-8f2d-b798e43529ad"). InnerVolumeSpecName "kube-api-access-fqjlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:35.561646 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.561598    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/458c6823-2a62-4357-8f2d-b798e43529ad-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:35.561646 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:35.561645    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqjlt\" (UniqueName: \"kubernetes.io/projected/458c6823-2a62-4357-8f2d-b798e43529ad-kube-api-access-fqjlt\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:36.009774 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.009733    2560 generic.go:356] "Generic (PLEG): container finished" podID="458c6823-2a62-4357-8f2d-b798e43529ad" containerID="6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b" exitCode=0
Jul 29 16:18:36.009951 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.009790    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"
Jul 29 16:18:36.009951 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.009817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4" event={"ID":"458c6823-2a62-4357-8f2d-b798e43529ad","Type":"ContainerDied","Data":"6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b"}
Jul 29 16:18:36.009951 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.009895    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4" event={"ID":"458c6823-2a62-4357-8f2d-b798e43529ad","Type":"ContainerDied","Data":"9a4225730c02d660b7772acef6ce234f536e1f3ec896463af24cfe12b9e430ce"}
Jul 29 16:18:36.009951 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.009920    2560 scope.go:117] "RemoveContainer" containerID="6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b"
Jul 29 16:18:36.019504 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.019481    2560 scope.go:117] "RemoveContainer" containerID="6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b"
Jul 29 16:18:36.019771 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:18:36.019750    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b\": container with ID starting with 6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b not found: ID does not exist" containerID="6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b"
Jul 29 16:18:36.019821 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.019780    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b"} err="failed to get container status \"6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b\": rpc error: code = NotFound desc = could not find container \"6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b\": container with ID starting with 6d14c85388a40a4f84dc76852abc9134cd29e736a386824dfcf1259b9e5c8d9b not found: ID does not exist"
Jul 29 16:18:36.032927 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.032895    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"]
Jul 29 16:18:36.040690 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.040660    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-16a6d7-8fb59c476-n7ft4"]
Jul 29 16:18:36.166701 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.166660    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="458c6823-2a62-4357-8f2d-b798e43529ad" path="/var/lib/kubelet/pods/458c6823-2a62-4357-8f2d-b798e43529ad/volumes"
Jul 29 16:18:36.167089 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:36.167075    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cceb4693-fe21-4ecd-93cb-a37021129f2a" path="/var/lib/kubelet/pods/cceb4693-fe21-4ecd-93cb-a37021129f2a/volumes"
Jul 29 16:18:38.490387 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:38.490346    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:18:53.475780 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.475737    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"]
Jul 29 16:18:53.479874 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.479841    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-02add2-29755695-pk2b5"]
Jul 29 16:18:53.530677 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.530635    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"]
Jul 29 16:18:53.530879 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.530853    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx" podUID="22ff2446-ad31-4d16-8dc3-66caeb4279e9" containerName="payload-pre-processing" containerID="cri-o://18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225" gracePeriod=30
Jul 29 16:18:53.788033 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.788006    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:18:53.814914 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.814883    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbnm6\" (UniqueName: \"kubernetes.io/projected/22ff2446-ad31-4d16-8dc3-66caeb4279e9-kube-api-access-lbnm6\") pod \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") "
Jul 29 16:18:53.815087 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.814935    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff2446-ad31-4d16-8dc3-66caeb4279e9-plugins-config-volume\") pod \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\" (UID: \"22ff2446-ad31-4d16-8dc3-66caeb4279e9\") "
Jul 29 16:18:53.815388 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.815362    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ff2446-ad31-4d16-8dc3-66caeb4279e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "22ff2446-ad31-4d16-8dc3-66caeb4279e9" (UID: "22ff2446-ad31-4d16-8dc3-66caeb4279e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:53.817088 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.817064    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ff2446-ad31-4d16-8dc3-66caeb4279e9-kube-api-access-lbnm6" (OuterVolumeSpecName: "kube-api-access-lbnm6") pod "22ff2446-ad31-4d16-8dc3-66caeb4279e9" (UID: "22ff2446-ad31-4d16-8dc3-66caeb4279e9"). InnerVolumeSpecName "kube-api-access-lbnm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:53.916509 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.916476    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbnm6\" (UniqueName: \"kubernetes.io/projected/22ff2446-ad31-4d16-8dc3-66caeb4279e9-kube-api-access-lbnm6\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:53.916509 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:53.916506    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff2446-ad31-4d16-8dc3-66caeb4279e9-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.116020 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.115971    2560 generic.go:356] "Generic (PLEG): container finished" podID="22ff2446-ad31-4d16-8dc3-66caeb4279e9" containerID="18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225" exitCode=0
Jul 29 16:18:54.116303 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.116029    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"
Jul 29 16:18:54.116303 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.116062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx" event={"ID":"22ff2446-ad31-4d16-8dc3-66caeb4279e9","Type":"ContainerDied","Data":"18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225"}
Jul 29 16:18:54.116303 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.116123    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx" event={"ID":"22ff2446-ad31-4d16-8dc3-66caeb4279e9","Type":"ContainerDied","Data":"112eec4f98d33a02f0962086b18a8b4c66295b01f34a041f0cb8cf2ba7072fde"}
Jul 29 16:18:54.116303 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.116146    2560 scope.go:117] "RemoveContainer" containerID="18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225"
Jul 29 16:18:54.127546 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.127524    2560 scope.go:117] "RemoveContainer" containerID="18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225"
Jul 29 16:18:54.127847 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:18:54.127827    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225\": container with ID starting with 18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225 not found: ID does not exist" containerID="18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225"
Jul 29 16:18:54.127913 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.127861    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225"} err="failed to get container status \"18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225\": rpc error: code = NotFound desc = could not find container \"18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225\": container with ID starting with 18089d7fa9078688b2351065f75c5890cf8d76788d5c277536f6a19506389225 not found: ID does not exist"
Jul 29 16:18:54.147566 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.147529    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"]
Jul 29 16:18:54.167173 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.167136    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b" path="/var/lib/kubelet/pods/2cbd4336-7cc1-4f8e-86c4-f9dce4393b8b/volumes"
Jul 29 16:18:54.167490 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:54.167470    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-02add2-6d599579c6-hknlx"]
Jul 29 16:18:56.166126 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:56.166089    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22ff2446-ad31-4d16-8dc3-66caeb4279e9" path="/var/lib/kubelet/pods/22ff2446-ad31-4d16-8dc3-66caeb4279e9/volumes"
Jul 29 16:18:56.988345 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:18:56.988306    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:19:11.812577 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:11.812541    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"]
Jul 29 16:19:11.817008 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:11.816979    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7e5daf-29755695-57chw"]
Jul 29 16:19:11.881659 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:11.881623    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"]
Jul 29 16:19:11.881902 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:11.881861    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2" podUID="8e8e49da-75c2-4885-8cbc-ff32bea703b5" containerName="payload-processing" containerID="cri-o://81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c" gracePeriod=30
Jul 29 16:19:11.889024 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:11.888993    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"]
Jul 29 16:19:11.889270 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:11.889240    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c" podUID="db4379bb-c8b1-4b1b-a952-5ebf48165c86" containerName="payload-pre-processing" containerID="cri-o://00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89" gracePeriod=30
Jul 29 16:19:12.133926 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.133902    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:19:12.165276 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.165245    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:19:12.167324 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.167298    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b81cb20-dcd1-416a-ad56-3963d3dac510" path="/var/lib/kubelet/pods/5b81cb20-dcd1-416a-ad56-3963d3dac510/volumes"
Jul 29 16:19:12.173391 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.173370    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e8e49da-75c2-4885-8cbc-ff32bea703b5-plugins-config-volume\") pod \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") "
Jul 29 16:19:12.173490 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.173426    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9wq4\" (UniqueName: \"kubernetes.io/projected/8e8e49da-75c2-4885-8cbc-ff32bea703b5-kube-api-access-h9wq4\") pod \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\" (UID: \"8e8e49da-75c2-4885-8cbc-ff32bea703b5\") "
Jul 29 16:19:12.173696 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.173675    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e8e49da-75c2-4885-8cbc-ff32bea703b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e8e49da-75c2-4885-8cbc-ff32bea703b5" (UID: "8e8e49da-75c2-4885-8cbc-ff32bea703b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:19:12.175419 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.175393    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e8e49da-75c2-4885-8cbc-ff32bea703b5-kube-api-access-h9wq4" (OuterVolumeSpecName: "kube-api-access-h9wq4") pod "8e8e49da-75c2-4885-8cbc-ff32bea703b5" (UID: "8e8e49da-75c2-4885-8cbc-ff32bea703b5"). InnerVolumeSpecName "kube-api-access-h9wq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:19:12.229054 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.229014    2560 generic.go:356] "Generic (PLEG): container finished" podID="8e8e49da-75c2-4885-8cbc-ff32bea703b5" containerID="81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c" exitCode=0
Jul 29 16:19:12.229281 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.229071    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"
Jul 29 16:19:12.229281 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.229100    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2" event={"ID":"8e8e49da-75c2-4885-8cbc-ff32bea703b5","Type":"ContainerDied","Data":"81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c"}
Jul 29 16:19:12.229281 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.229176    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2" event={"ID":"8e8e49da-75c2-4885-8cbc-ff32bea703b5","Type":"ContainerDied","Data":"c383f31b90f14ac3b482bf3e9e4f8ce5da88e0e9996f76d6f3ce701d0ca18de8"}
Jul 29 16:19:12.229281 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.229234    2560 scope.go:117] "RemoveContainer" containerID="81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c"
Jul 29 16:19:12.230624 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.230596    2560 generic.go:356] "Generic (PLEG): container finished" podID="db4379bb-c8b1-4b1b-a952-5ebf48165c86" containerID="00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89" exitCode=0
Jul 29 16:19:12.230769 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.230665    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"
Jul 29 16:19:12.230769 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.230680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c" event={"ID":"db4379bb-c8b1-4b1b-a952-5ebf48165c86","Type":"ContainerDied","Data":"00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89"}
Jul 29 16:19:12.230769 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.230734    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c" event={"ID":"db4379bb-c8b1-4b1b-a952-5ebf48165c86","Type":"ContainerDied","Data":"802d6d303cb618ffc45f3f2827aed20741ba280c82c204a6251241b6e283ca46"}
Jul 29 16:19:12.242465 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.242445    2560 scope.go:117] "RemoveContainer" containerID="81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c"
Jul 29 16:19:12.242752 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:19:12.242724    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c\": container with ID starting with 81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c not found: ID does not exist" containerID="81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c"
Jul 29 16:19:12.242830 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.242762    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c"} err="failed to get container status \"81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c\": rpc error: code = NotFound desc = could not find container \"81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c\": container with ID starting with 81aa6e977f5818cb0ca6f45290bda56c1186e1489a483df9229c21980f66c63c not found: ID does not exist"
Jul 29 16:19:12.242830 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.242787    2560 scope.go:117] "RemoveContainer" containerID="00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89"
Jul 29 16:19:12.251853 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.251833    2560 scope.go:117] "RemoveContainer" containerID="00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89"
Jul 29 16:19:12.252123 ip-10-0-136-199 kubenswrapper[2560]: E0729 16:19:12.252103    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89\": container with ID starting with 00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89 not found: ID does not exist" containerID="00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89"
Jul 29 16:19:12.252212 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.252132    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89"} err="failed to get container status \"00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89\": rpc error: code = NotFound desc = could not find container \"00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89\": container with ID starting with 00090862cea83d176b1750457b72ecf24f228b07f0456063eac88d5ebfbbcf89 not found: ID does not exist"
Jul 29 16:19:12.257549 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.257524    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"]
Jul 29 16:19:12.262674 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.262650    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7e5daf-55446ff8fd-vxgw2"]
Jul 29 16:19:12.274575 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.274553    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngcz5\" (UniqueName: \"kubernetes.io/projected/db4379bb-c8b1-4b1b-a952-5ebf48165c86-kube-api-access-ngcz5\") pod \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") "
Jul 29 16:19:12.274714 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.274697    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4379bb-c8b1-4b1b-a952-5ebf48165c86-plugins-config-volume\") pod \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\" (UID: \"db4379bb-c8b1-4b1b-a952-5ebf48165c86\") "
Jul 29 16:19:12.275021 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.275005    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e8e49da-75c2-4885-8cbc-ff32bea703b5-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:19:12.275064 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.275026    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9wq4\" (UniqueName: \"kubernetes.io/projected/8e8e49da-75c2-4885-8cbc-ff32bea703b5-kube-api-access-h9wq4\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:19:12.275098 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.275066    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db4379bb-c8b1-4b1b-a952-5ebf48165c86-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db4379bb-c8b1-4b1b-a952-5ebf48165c86" (UID: "db4379bb-c8b1-4b1b-a952-5ebf48165c86"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:19:12.276782 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.276764    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4379bb-c8b1-4b1b-a952-5ebf48165c86-kube-api-access-ngcz5" (OuterVolumeSpecName: "kube-api-access-ngcz5") pod "db4379bb-c8b1-4b1b-a952-5ebf48165c86" (UID: "db4379bb-c8b1-4b1b-a952-5ebf48165c86"). InnerVolumeSpecName "kube-api-access-ngcz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:19:12.376074 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.375972    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4379bb-c8b1-4b1b-a952-5ebf48165c86-plugins-config-volume\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:19:12.376074 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.376016    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngcz5\" (UniqueName: \"kubernetes.io/projected/db4379bb-c8b1-4b1b-a952-5ebf48165c86-kube-api-access-ngcz5\") on node \"ip-10-0-136-199.ec2.internal\" DevicePath \"\""
Jul 29 16:19:12.558079 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.558045    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"]
Jul 29 16:19:12.567079 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:12.567051    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7e5daf-6bf87d4cb8-4tm4c"]
Jul 29 16:19:14.167044 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:14.167009    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e8e49da-75c2-4885-8cbc-ff32bea703b5" path="/var/lib/kubelet/pods/8e8e49da-75c2-4885-8cbc-ff32bea703b5/volumes"
Jul 29 16:19:14.167453 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:14.167397    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db4379bb-c8b1-4b1b-a952-5ebf48165c86" path="/var/lib/kubelet/pods/db4379bb-c8b1-4b1b-a952-5ebf48165c86/volumes"
Jul 29 16:19:15.079679 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:15.079629    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggt7p"]
Jul 29 16:19:19.559042 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:19.558996    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cb6d9696c-276l5_a66ef0de-8bb6-49d2-8f8f-defa225bc09c/authorino/0.log"
Jul 29 16:19:26.013149 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:26.013116    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-lpwcm_736f8df5-cf77-4d6b-97d9-403811519e1a/copy-manifests/0.log"
Jul 29 16:19:26.022952 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:26.022905    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-lpwcm_736f8df5-cf77-4d6b-97d9-403811519e1a/manager/0.log"
Jul 29 16:19:26.650551 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:26.650520    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755665-vltfb_d2f681ae-4558-4d3f-a30e-f268b3ad8517/cleanup/0.log"
Jul 29 16:19:26.881108 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:26.881071    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755695-8vlkt_405fda58-82b9-4d07-9509-75fff1df213e/cleanup/0.log"
Jul 29 16:19:27.805370 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:27.805343    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx_547b007a-f31c-4dd2-bcc8-1bdddfbc6007/util/0.log"
Jul 29 16:19:27.814362 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:27.814330    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx_547b007a-f31c-4dd2-bcc8-1bdddfbc6007/pull/0.log"
Jul 29 16:19:27.822137 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:27.822113    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx_547b007a-f31c-4dd2-bcc8-1bdddfbc6007/extract/0.log"
Jul 29 16:19:27.944997 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:27.944961    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5_5bb2d649-1503-4f11-9513-a54f19a896a6/util/0.log"
Jul 29 16:19:27.953235 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:27.953207    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5_5bb2d649-1503-4f11-9513-a54f19a896a6/pull/0.log"
Jul 29 16:19:27.963198 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:27.963158    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5_5bb2d649-1503-4f11-9513-a54f19a896a6/extract/0.log"
Jul 29 16:19:28.083899 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.083825    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6_90e935cf-9678-45d1-a058-ffa367641195/extract/0.log"
Jul 29 16:19:28.094987 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.094958    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6_90e935cf-9678-45d1-a058-ffa367641195/util/0.log"
Jul 29 16:19:28.106735 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.106709    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6_90e935cf-9678-45d1-a058-ffa367641195/pull/0.log"
Jul 29 16:19:28.226981 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.226953    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt_e97debba-fa28-4455-9582-e282c56a4ac0/util/0.log"
Jul 29 16:19:28.240085 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.240035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt_e97debba-fa28-4455-9582-e282c56a4ac0/pull/0.log"
Jul 29 16:19:28.248777 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.248749    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt_e97debba-fa28-4455-9582-e282c56a4ac0/extract/0.log"
Jul 29 16:19:28.382931 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.382897    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cb6d9696c-276l5_a66ef0de-8bb6-49d2-8f8f-defa225bc09c/authorino/0.log"
Jul 29 16:19:28.624068 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:28.624037    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s4lsc_29753784-4887-4355-8a7e-eff29ab18e9b/manager/0.log"
Jul 29 16:19:29.117856 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:29.117829    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ggt7p_ecda580b-18a1-435d-bb7a-44f97d6afec4/limitador/0.log"
Jul 29 16:19:29.874743 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:29.874710    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-hbsc7_35d05b49-f710-4eb3-9e29-650cb38ec6a1/kube-auth-proxy/0.log"
Jul 29 16:19:31.092164 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:31.092121    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mwkhx_45bb1207-576a-448c-981b-3f4326059884/main/0.log"
Jul 29 16:19:31.457806 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:31.457724    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-dh9vt_e9339091-fba0-4dd6-8d99-dd054b9c5391/main/0.log"
Jul 29 16:19:37.136905 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.136812    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6xxwd/must-gather-bsf95"]
Jul 29 16:19:37.137451 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137426    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cceb4693-fe21-4ecd-93cb-a37021129f2a" containerName="revoke-keys"
Jul 29 16:19:37.137498 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137456    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cceb4693-fe21-4ecd-93cb-a37021129f2a" containerName="revoke-keys"
Jul 29 16:19:37.137498 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137479    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4379bb-c8b1-4b1b-a952-5ebf48165c86" containerName="payload-pre-processing"
Jul 29 16:19:37.137498 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137488    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4379bb-c8b1-4b1b-a952-5ebf48165c86" containerName="payload-pre-processing"
Jul 29 16:19:37.137593 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137501    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458c6823-2a62-4357-8f2d-b798e43529ad" containerName="payload-pre-processing"
Jul 29 16:19:37.137593 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137510    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="458c6823-2a62-4357-8f2d-b798e43529ad" containerName="payload-pre-processing"
Jul 29 16:19:37.137593 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137525    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e8e49da-75c2-4885-8cbc-ff32bea703b5" containerName="payload-processing"
Jul 29 16:19:37.137593 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137534    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e8e49da-75c2-4885-8cbc-ff32bea703b5" containerName="payload-processing"
Jul 29 16:19:37.137593 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137571    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22ff2446-ad31-4d16-8dc3-66caeb4279e9" containerName="payload-pre-processing"
Jul 29 16:19:37.137593 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137579    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="22ff2446-ad31-4d16-8dc3-66caeb4279e9" containerName="payload-pre-processing"
Jul 29 16:19:37.137794 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137683    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4379bb-c8b1-4b1b-a952-5ebf48165c86" containerName="payload-pre-processing"
Jul 29 16:19:37.137794 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137698    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cceb4693-fe21-4ecd-93cb-a37021129f2a" containerName="revoke-keys"
Jul 29 16:19:37.137794 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137709    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="458c6823-2a62-4357-8f2d-b798e43529ad" containerName="payload-pre-processing"
Jul 29 16:19:37.137794 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137720    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="22ff2446-ad31-4d16-8dc3-66caeb4279e9" containerName="payload-pre-processing"
Jul 29 16:19:37.137794 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.137729    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e8e49da-75c2-4885-8cbc-ff32bea703b5" containerName="payload-processing"
Jul 29 16:19:37.141220 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.141199    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.143776 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.143755    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6xxwd\"/\"openshift-service-ca.crt\""
Jul 29 16:19:37.143890 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.143755    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6xxwd\"/\"kube-root-ca.crt\""
Jul 29 16:19:37.144378 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.144364    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6xxwd\"/\"default-dockercfg-hmbm7\""
Jul 29 16:19:37.151574 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.151552    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xxwd/must-gather-bsf95"]
Jul 29 16:19:37.297133 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.297093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n65kp\" (UniqueName: \"kubernetes.io/projected/eda5e6e6-ba98-49a6-a372-ce69ecc51e5c-kube-api-access-n65kp\") pod \"must-gather-bsf95\" (UID: \"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c\") " pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.297368 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.297149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eda5e6e6-ba98-49a6-a372-ce69ecc51e5c-must-gather-output\") pod \"must-gather-bsf95\" (UID: \"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c\") " pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.398261 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.398138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eda5e6e6-ba98-49a6-a372-ce69ecc51e5c-must-gather-output\") pod \"must-gather-bsf95\" (UID: \"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c\") " pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.398261 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.398257    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n65kp\" (UniqueName: \"kubernetes.io/projected/eda5e6e6-ba98-49a6-a372-ce69ecc51e5c-kube-api-access-n65kp\") pod \"must-gather-bsf95\" (UID: \"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c\") " pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.398571 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.398548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eda5e6e6-ba98-49a6-a372-ce69ecc51e5c-must-gather-output\") pod \"must-gather-bsf95\" (UID: \"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c\") " pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.408229 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.408194    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n65kp\" (UniqueName: \"kubernetes.io/projected/eda5e6e6-ba98-49a6-a372-ce69ecc51e5c-kube-api-access-n65kp\") pod \"must-gather-bsf95\" (UID: \"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c\") " pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.451309 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.451257    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xxwd/must-gather-bsf95"
Jul 29 16:19:37.591260 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.591233    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xxwd/must-gather-bsf95"]
Jul 29 16:19:37.593063 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:19:37.593030    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeda5e6e6_ba98_49a6_a372_ce69ecc51e5c.slice/crio-50de05b41fa7e79947f7de42ade9c40c79ce9d198d5e12183c0631dca5a37a61 WatchSource:0}: Error finding container 50de05b41fa7e79947f7de42ade9c40c79ce9d198d5e12183c0631dca5a37a61: Status 404 returned error can't find the container with id 50de05b41fa7e79947f7de42ade9c40c79ce9d198d5e12183c0631dca5a37a61
Jul 29 16:19:37.595012 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:37.594991    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:19:38.377945 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:38.377911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xxwd/must-gather-bsf95" event={"ID":"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c","Type":"ContainerStarted","Data":"50de05b41fa7e79947f7de42ade9c40c79ce9d198d5e12183c0631dca5a37a61"}
Jul 29 16:19:39.388771 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:39.388736    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xxwd/must-gather-bsf95" event={"ID":"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c","Type":"ContainerStarted","Data":"de4fb1ab51ff0a3ebe7e9d9a3478b566dc3139437347cb2ae98c88640138b5e9"}
Jul 29 16:19:39.389303 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:39.388782    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xxwd/must-gather-bsf95" event={"ID":"eda5e6e6-ba98-49a6-a372-ce69ecc51e5c","Type":"ContainerStarted","Data":"69d364c660dbc9e24fdfc24d6d35ebb2d2f57657ead3515420deed799cb94756"}
Jul 29 16:19:39.406919 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:39.406792    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6xxwd/must-gather-bsf95" podStartSLOduration=1.286105999 podStartE2EDuration="2.406772305s" totalImagesPullingTime="1.120666306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:19:39.404865835 +0000 UTC m=+4117.698802638" watchObservedRunningTime="2026-07-29 16:19:39.406772305 +0000 UTC m=+4117.700709108"
Jul 29 16:19:40.447358 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:40.447323    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vvz9k_025b10db-422e-4bbc-b0b2-e4dfa0c5fecf/global-pull-secret-syncer/0.log"
Jul 29 16:19:40.568891 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:40.568863    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vd2hl_358e0c8d-7a8b-4876-9b32-2c379aeccecb/konnectivity-agent/0.log"
Jul 29 16:19:40.687090 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:40.687039    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-199.ec2.internal_cf62266a6ae3b77620eda580b89764b9/haproxy/0.log"
Jul 29 16:19:44.453252 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.453217    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx_547b007a-f31c-4dd2-bcc8-1bdddfbc6007/extract/0.log"
Jul 29 16:19:44.488019 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.487984    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx_547b007a-f31c-4dd2-bcc8-1bdddfbc6007/util/0.log"
Jul 29 16:19:44.518842 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.518805    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759l4hrx_547b007a-f31c-4dd2-bcc8-1bdddfbc6007/pull/0.log"
Jul 29 16:19:44.555110 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.555073    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5_5bb2d649-1503-4f11-9513-a54f19a896a6/extract/0.log"
Jul 29 16:19:44.586816 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.586761    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5_5bb2d649-1503-4f11-9513-a54f19a896a6/util/0.log"
Jul 29 16:19:44.617351 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.617287    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hpvt5_5bb2d649-1503-4f11-9513-a54f19a896a6/pull/0.log"
Jul 29 16:19:44.677679 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.677640    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6_90e935cf-9678-45d1-a058-ffa367641195/extract/0.log"
Jul 29 16:19:44.703172 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.703143    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6_90e935cf-9678-45d1-a058-ffa367641195/util/0.log"
Jul 29 16:19:44.732428 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.732323    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8sb6_90e935cf-9678-45d1-a058-ffa367641195/pull/0.log"
Jul 29 16:19:44.779384 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.779357    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt_e97debba-fa28-4455-9582-e282c56a4ac0/extract/0.log"
Jul 29 16:19:44.809786 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.809755    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt_e97debba-fa28-4455-9582-e282c56a4ac0/util/0.log"
Jul 29 16:19:44.837890 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:44.837863    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12kgvt_e97debba-fa28-4455-9582-e282c56a4ac0/pull/0.log"
Jul 29 16:19:45.619228 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:45.619111    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cb6d9696c-276l5_a66ef0de-8bb6-49d2-8f8f-defa225bc09c/authorino/0.log"
Jul 29 16:19:45.700815 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:45.700780    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s4lsc_29753784-4887-4355-8a7e-eff29ab18e9b/manager/0.log"
Jul 29 16:19:45.905049 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:45.904956    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ggt7p_ecda580b-18a1-435d-bb7a-44f97d6afec4/limitador/0.log"
Jul 29 16:19:47.897361 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:47.897325    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/1.log"
Jul 29 16:19:48.061205 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.060898    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ccv7r_d8b85fba-8719-4dee-a1b5-e94d9747836b/cluster-monitoring-operator/0.log"
Jul 29 16:19:48.099475 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.099363    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ztszl_89eff43c-1c38-408c-8338-2357ac807bbf/kube-state-metrics/0.log"
Jul 29 16:19:48.123744 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.123711    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ztszl_89eff43c-1c38-408c-8338-2357ac807bbf/kube-rbac-proxy-main/0.log"
Jul 29 16:19:48.156652 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.156568    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ztszl_89eff43c-1c38-408c-8338-2357ac807bbf/kube-rbac-proxy-self/0.log"
Jul 29 16:19:48.196449 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.196419    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-67bf84cdff-rkndn_d031dd99-2750-4002-be09-c09655f5f6bf/metrics-server/0.log"
Jul 29 16:19:48.443445 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.443350    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s2ddf_5483010b-120e-40b4-874b-ce9e21436739/node-exporter/0.log"
Jul 29 16:19:48.471022 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.470936    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s2ddf_5483010b-120e-40b4-874b-ce9e21436739/kube-rbac-proxy/0.log"
Jul 29 16:19:48.498281 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.498249    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s2ddf_5483010b-120e-40b4-874b-ce9e21436739/init-textfile/0.log"
Jul 29 16:19:48.537717 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.537679    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-gcd5g_c2027a2a-60f4-443d-8ef2-e2917c68c983/kube-rbac-proxy-main/0.log"
Jul 29 16:19:48.563035 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.562991    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-gcd5g_c2027a2a-60f4-443d-8ef2-e2917c68c983/kube-rbac-proxy-self/0.log"
Jul 29 16:19:48.589474 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.589443    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-gcd5g_c2027a2a-60f4-443d-8ef2-e2917c68c983/openshift-state-metrics/0.log"
Jul 29 16:19:48.882178 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:48.881997    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-mmdqn_69316d5f-c98d-43a6-95ed-de7442c232dd/prometheus-operator-admission-webhook/0.log"
Jul 29 16:19:50.343307 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.343269    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-rrds9_6875b039-7cc3-425b-a5aa-c45f0d63d01a/networking-console-plugin/0.log"
Jul 29 16:19:50.882216 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.882143    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"]
Jul 29 16:19:50.887782 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.887756    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:50.894289 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.894258    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"]
Jul 29 16:19:50.932933 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.932887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-podres\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:50.932933 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.932936    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-lib-modules\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:50.933176 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.933018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-sys\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:50.933176 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.933054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-proc\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:50.933176 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:50.933092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkbqs\" (UniqueName: \"kubernetes.io/projected/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-kube-api-access-qkbqs\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034260 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-sys\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034589 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-proc\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034711 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-sys\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034711 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkbqs\" (UniqueName: \"kubernetes.io/projected/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-kube-api-access-qkbqs\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034711 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034670    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-proc\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034879 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-podres\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034879 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-lib-modules\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034982 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034931    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-podres\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.034982 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.034955    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-lib-modules\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.045755 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.045726    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkbqs\" (UniqueName: \"kubernetes.io/projected/0f3972c1-209f-4eeb-a5c9-4e808bf269e1-kube-api-access-qkbqs\") pod \"perf-node-gather-daemonset-qn6tr\" (UID: \"0f3972c1-209f-4eeb-a5c9-4e808bf269e1\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.202686 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.202588    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.383729 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.383701    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"]
Jul 29 16:19:51.386720 ip-10-0-136-199 kubenswrapper[2560]: W0729 16:19:51.386690    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0f3972c1_209f_4eeb_a5c9_4e808bf269e1.slice/crio-6cb44134b8195b25d1d5925aa9deae64ad74fd1f4142f3311bf7e994f4468cf3 WatchSource:0}: Error finding container 6cb44134b8195b25d1d5925aa9deae64ad74fd1f4142f3311bf7e994f4468cf3: Status 404 returned error can't find the container with id 6cb44134b8195b25d1d5925aa9deae64ad74fd1f4142f3311bf7e994f4468cf3
Jul 29 16:19:51.508153 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.508111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr" event={"ID":"0f3972c1-209f-4eeb-a5c9-4e808bf269e1","Type":"ContainerStarted","Data":"39532fc26f0373f9f4e048040cd4ddfaec8291a20024471dea7fd186e82d5257"}
Jul 29 16:19:51.508383 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.508213    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:19:51.508383 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.508246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr" event={"ID":"0f3972c1-209f-4eeb-a5c9-4e808bf269e1","Type":"ContainerStarted","Data":"6cb44134b8195b25d1d5925aa9deae64ad74fd1f4142f3311bf7e994f4468cf3"}
Jul 29 16:19:51.531892 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.531822    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr" podStartSLOduration=1.531797488 podStartE2EDuration="1.531797488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 16:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:19:51.529427615 +0000 UTC m=+4129.823364419" watchObservedRunningTime="2026-07-29 16:19:51.531797488 +0000 UTC m=+4129.825734291"
Jul 29 16:19:51.688147 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.688113    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-857fdfbb7f-bbhqn_705f19d6-9d73-45b3-bedd-d981f460bbcc/console/0.log"
Jul 29 16:19:51.746727 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:51.746638    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-dwq45_7929f79e-0591-475b-aa42-17f4bc2b24d2/download-server/0.log"
Jul 29 16:19:52.385020 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:52.384982    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-ctnd2_a6fbfe9b-c402-42c5-a0c1-258a123a5821/volume-data-source-validator/0.log"
Jul 29 16:19:53.536410 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:53.536371    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h79ts_bdf8ca9f-0611-463b-b647-e7809c88f8c1/dns/0.log"
Jul 29 16:19:53.562843 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:53.562808    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h79ts_bdf8ca9f-0611-463b-b647-e7809c88f8c1/kube-rbac-proxy/0.log"
Jul 29 16:19:53.675967 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:53.675935    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6z2cx_35b71621-959c-4601-9800-f5e3f26182fa/dns-node-resolver/0.log"
Jul 29 16:19:54.332806 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:54.332764    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6b67r_52224984-89d4-44bb-8119-2a95db6c1d03/node-ca/0.log"
Jul 29 16:19:55.693046 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:55.692988    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-hbsc7_35d05b49-f710-4eb3-9e29-650cb38ec6a1/kube-auth-proxy/0.log"
Jul 29 16:19:56.740724 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:56.740684    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dpl5l_c46543f0-2496-4e4d-9a40-4cc67663aedf/serve-healthcheck-canary/0.log"
Jul 29 16:19:57.314447 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:57.314415    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7p52g_caed7015-d329-411d-891e-69649ef42740/kube-rbac-proxy/0.log"
Jul 29 16:19:57.341547 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:57.341513    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7p52g_caed7015-d329-411d-891e-69649ef42740/exporter/0.log"
Jul 29 16:19:57.376445 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:57.376389    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7p52g_caed7015-d329-411d-891e-69649ef42740/extractor/0.log"
Jul 29 16:19:57.525607 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:19:57.525577    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-qn6tr"
Jul 29 16:20:00.479221 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:00.479145    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-lpwcm_736f8df5-cf77-4d6b-97d9-403811519e1a/manager/0.log"
Jul 29 16:20:00.505581 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:00.505554    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-lpwcm_736f8df5-cf77-4d6b-97d9-403811519e1a/copy-manifests/0.log"
Jul 29 16:20:01.649245 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:01.649218    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-hbsc7_35d05b49-f710-4eb3-9e29-650cb38ec6a1/kube-auth-proxy/0.log"
Jul 29 16:20:01.939721 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:01.939637    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-9bd7bffd8-wnp4m_6f3b4bd5-1356-498b-a787-fea41270f146/manager/0.log"
Jul 29 16:20:03.572761 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:03.572730    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-hbsc7_35d05b49-f710-4eb3-9e29-650cb38ec6a1/kube-auth-proxy/0.log"
Jul 29 16:20:07.238988 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:07.238954    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-nh6zz_885663bb-5d2f-4c90-b7f8-1004f54cd1a8/migrator/0.log"
Jul 29 16:20:07.266409 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:07.266379    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-nh6zz_885663bb-5d2f-4c90-b7f8-1004f54cd1a8/graceful-termination/0.log"
Jul 29 16:20:07.653150 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:07.653108    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5_96198947-8a41-49ad-baeb-c8d609ef2cb8/kube-storage-version-migrator-operator/1.log"
Jul 29 16:20:07.655065 ip-10-0-136-199 kubenswrapper[2560]: I0729 16:20:07.655036    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-dpgg5_96198947-8a41-49ad-baeb-c8d609ef2cb8/kube-storage-version-migrator-operator/0.log"