Aug 04 09:07:47.253097 ip-10-0-143-106 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 09:07:47.701995 ip-10-0-143-106 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 09:07:47.701995 ip-10-0-143-106 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 09:07:47.701995 ip-10-0-143-106 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 09:07:47.702815 ip-10-0-143-106 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 09:07:47.702815 ip-10-0-143-106 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 09:07:47.702815 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.702186    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 09:07:47.707271 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707256    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 09:07:47.707271 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707270    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 09:07:47.707271 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707273    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707276    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707279    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707282    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707284    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707287    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707289    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707292    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707295    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707297    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707300    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707302    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707305    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707307    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707310    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707312    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707316    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707319    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707321    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707324    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 09:07:47.707357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707326    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707329    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707331    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707334    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707337    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707340    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707342    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707352    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707355    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707358    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707361    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707364    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707367    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707371    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707376    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707379    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707383    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707386    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707389    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707391    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 09:07:47.707852 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707394    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707396    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707400    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707403    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707406    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707409    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707412    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707415    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707417    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707420    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707423    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707426    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707428    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707431    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707436    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707438    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707441    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707444    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707446    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707449    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 09:07:47.708333 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707451    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707454    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707456    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707459    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707461    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707464    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707466    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707468    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707471    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707473    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707476    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707478    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707483    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707487    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707489    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707492    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707494    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707496    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707499    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707502    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 09:07:47.708884 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707505    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707508    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707511    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707513    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707515    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707519    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707522    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707524    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707527    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707529    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707532    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707535    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707537    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707540    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707542    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707545    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 09:07:47.709359 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.707548    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 09:07:47.709996 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.709983    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.709998    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710002    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710005    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710009    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710012    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710016    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710020    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710023    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710025    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710028    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710031    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710033    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710036    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710039    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710042    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710044    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710047    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710049    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710052    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 09:07:47.710068 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710055    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710057    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710060    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710062    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710065    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710068    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710071    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710073    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710077    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710079    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710082    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710084    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710087    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710089    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710092    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710094    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710097    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710099    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710102    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710104    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 09:07:47.710559 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710107    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710109    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710111    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710114    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710116    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710119    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710121    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710124    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710127    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710129    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710131    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710134    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710136    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710141    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710145    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710149    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710152    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710154    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710157    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 09:07:47.711134 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710160    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710162    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710165    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710167    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710169    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710172    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710174    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710177    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710180    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710183    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710185    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710188    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710191    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710193    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710195    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710198    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710200    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710203    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710205    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710208    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 09:07:47.711586 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710210    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710213    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710215    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710218    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710220    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710223    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710226    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710229    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710231    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710234    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710236    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710239    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710241    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710244    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710246    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710249    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710251    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710254    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710256    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710259    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 09:07:47.712092 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710620    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710625    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710628    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710632    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710634    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710637    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710639    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710642    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710644    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710647    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710649    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710651    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710654    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710657    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710659    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710663    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710665    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710668    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710670    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 09:07:47.712568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710673    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710676    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710678    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710681    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710683    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710686    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710688    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710690    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710692    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710695    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710697    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710700    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710702    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710704    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710707    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710709    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710711    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710714    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710716    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710718    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 09:07:47.713069 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710721    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710723    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710725    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710728    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710730    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710732    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710735    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710738    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710741    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710745    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710749    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710752    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710756    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710759    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710762    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710765    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710767    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710770    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710793    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710796    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 09:07:47.713569 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710799    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710801    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710805    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710808    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710811    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710814    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710817    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710820    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710822    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710825    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710827    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710830    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710832    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710834    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710837    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710839    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710842    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710845    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710847    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 09:07:47.714158 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710849    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710853    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710858    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710861    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710863    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710866    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710868    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710871    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710873    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710875    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710878    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710881    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710883    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710885    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710888    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710890    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710892    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710895    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710897    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710899    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 09:07:47.714620 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.710902    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.710973    2558 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.710980    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.710987    2558 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.710991    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.710995    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711000    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711006    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711012    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711017    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711022    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711026    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711029    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711034    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711045    2558 flags.go:64] FLAG: --cgroup-root=""
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711048    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711050    2558 flags.go:64] FLAG: --client-ca-file=""
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711053    2558 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711056    2558 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711060    2558 flags.go:64] FLAG: --cluster-domain=""
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711063    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711066    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711069    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711072    2558 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 09:07:47.715118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711082    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711085    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711088    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711091    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711093    2558 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711096    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711100    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711103    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711105    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711109    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711112    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711115    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711118    2558 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711121    2558 flags.go:64] FLAG: --enable-server="true"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711124    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711131    2558 flags.go:64] FLAG: --event-burst="100"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711133    2558 flags.go:64] FLAG: --event-qps="50"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711137    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711140    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711143    2558 flags.go:64] FLAG: --eviction-hard=""
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711146    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711151    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711154    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711157    2558 flags.go:64] FLAG: --eviction-soft=""
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711160    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 09:07:47.715676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711163    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711165    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711168    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711171    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711174    2558 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711176    2558 flags.go:64] FLAG: --feature-gates=""
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711184    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711187    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711190    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711193    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711195    2558 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711198    2558 flags.go:64] FLAG: --help="false"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711201    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-143-106.ec2.internal"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711204    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711207    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711210    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711213    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711216    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711219    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711235    2558 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711238    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711241    2558 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711244    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711247    2558 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 09:07:47.716331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711250    2558 flags.go:64] FLAG: --kube-reserved=""
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711253    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711256    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711259    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711263    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711266    2558 flags.go:64] FLAG: --lock-file=""
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711268    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711272    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711274    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711280    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711283    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711285    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711288    2558 flags.go:64] FLAG: --logging-format="text"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711291    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711294    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711296    2558 flags.go:64] FLAG: --manifest-url=""
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711299    2558 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711303    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711306    2558 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711310    2558 flags.go:64] FLAG: --max-pods="110"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711313    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711316    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711318    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711321    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711324    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 09:07:47.716958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711327    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711329    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711336    2558 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711339    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711341    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711344    2558 flags.go:64] FLAG: --pod-cidr=""
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711348    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711354    2558 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711356    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711359    2558 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711362    2558 flags.go:64] FLAG: --port="10250"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711365    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711369    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e98b61af89f8b061"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711372    2558 flags.go:64] FLAG: --qos-reserved=""
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711375    2558 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711378    2558 flags.go:64] FLAG: --register-node="true"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711381    2558 flags.go:64] FLAG: --register-with-taints=""
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711384    2558 flags.go:64] FLAG: --registry-burst="10"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711387    2558 flags.go:64] FLAG: --registry-qps="5"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711390    2558 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711392    2558 flags.go:64] FLAG: --reserved-memory=""
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711396    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711399    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711402    2558 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711404    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 09:07:47.717559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711407    2558 flags.go:64] FLAG: --runonce="false"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711410    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711413    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711416    2558 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711418    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711421    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711424    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711427    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711430    2558 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711433    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711435    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711438    2558 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711441    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711445    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711448    2558 flags.go:64] FLAG: --system-cgroups=""
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711451    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711456    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711459    2558 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711461    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711467    2558 flags.go:64] FLAG: --tls-min-version=""
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711469    2558 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711472    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711475    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711479    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711481    2558 flags.go:64] FLAG: --v="2"
Aug 04 09:07:47.718180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711485    2558 flags.go:64] FLAG: --version="false"
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711489    2558 flags.go:64] FLAG: --vmodule=""
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711492    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.711496    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711602    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711605    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711608    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711611    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711614    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711616    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711619    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711621    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711624    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711627    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711629    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711631    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711634    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711637    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711640    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711642    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 09:07:47.718844 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711646    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711649    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711651    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711654    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711656    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711659    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711664    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711666    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711669    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711671    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711675    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711679    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711683    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711687    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711690    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711693    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711696    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711699    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711701    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 09:07:47.719328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711704    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711707    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711709    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711712    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711714    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711717    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711719    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711721    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711724    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711726    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711729    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711731    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711733    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711737    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711739    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711742    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711744    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711747    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711749    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711753    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 09:07:47.719794 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711755    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711758    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711760    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711764    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711767    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711784    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711788    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711790    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711793    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711796    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711798    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711801    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711803    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711805    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711808    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711811    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711813    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711816    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711818    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 09:07:47.720298 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711821    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711823    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711826    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711828    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711831    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711833    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711837    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711840    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711842    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711844    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711847    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711850    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711854    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711856    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711859    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711861    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711863    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711866    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711868    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711871    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 09:07:47.720806 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711874    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 09:07:47.721328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711876    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 09:07:47.721328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711879    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 09:07:47.721328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711881    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 09:07:47.721328 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.711884    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 09:07:47.721328 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.712626    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 09:07:47.721328 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.720674    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 09:07:47.721328 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.720689    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 09:07:47.721591 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.721580    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 09:07:47.721619 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.721591    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 09:07:47.721655 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721648    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 09:07:47.721655 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721655    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721658    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721661    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721664    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721666    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721669    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721672    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721674    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721677    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721681    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721684    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721688    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721691    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721694    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721697    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721699    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721702    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721705    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721707    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 09:07:47.721711 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721710    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721713    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721716    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721719    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721722    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721725    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721727    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721730    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721732    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721735    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721737    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721740    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721742    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721744    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721747    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721749    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721752    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721754    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721757    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721759    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 09:07:47.722187 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721762    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721765    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721767    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721769    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721782    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721785    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721788    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721790    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721793    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721795    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721798    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721800    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721803    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721806    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721810    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721813    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721816    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721818    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721821    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721824    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 09:07:47.722675 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721827    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721829    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721832    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721834    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721837    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721840    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721842    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721845    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721848    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721850    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721852    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721856    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721858    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721861    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721863    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721866    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721868    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721870    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721873    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721875    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 09:07:47.723181 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721878    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721881    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721883    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721886    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721890    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721893    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721897    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721900    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721903    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721907    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721909    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721912    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721915    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721917    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721919    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721922    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721924    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721927    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 09:07:47.723692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.721929    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.721934    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722042    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722046    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722050    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722052    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722056    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722060    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722063    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722066    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722068    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722071    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722074    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722077    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 09:07:47.724193 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722080    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722083    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722085    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722087    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722090    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722093    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722096    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722098    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722101    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722104    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722106    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722109    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722111    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722114    2558 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722116    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722119    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722121    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722124    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722127    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722129    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 09:07:47.724540 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722132    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722134    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722137    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722140    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722142    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722144    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722147    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722149    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722152    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722154    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722157    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722160    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722162    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722165    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722167    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722170    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722172    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722175    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722177    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722180    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 09:07:47.725057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722182    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722186    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722189    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722191    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722194    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722196    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722199    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722202    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722204    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722207    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722209    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722213    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722216    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722219    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722222    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722224    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722227    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722230    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722232    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 09:07:47.725531 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722235    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722237    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722240    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722242    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722244    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722247    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722249    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722252    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722254    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722257    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722259    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722262    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722264    2558 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722267    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722269    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722272    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722274    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722277    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722279    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722282    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 09:07:47.726057 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722284    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722286    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722289    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722291    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722293    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722296    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722298    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:47.722301    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.722306    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.722400    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 09:07:47.726541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.725653    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 09:07:47.727291 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.727279    2558 server.go:1019] "Starting client certificate rotation"
Aug 04 09:07:47.727415 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.727396    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 09:07:47.727466 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.727447    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 09:07:47.730128 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.730110    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 09:07:47.731219 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.731201    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 09:07:47.742368 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.742351    2558 log.go:25] "Validated CRI v1 runtime API"
Aug 04 09:07:47.745579 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.745560    2558 log.go:25] "Validated CRI v1 image API"
Aug 04 09:07:47.746421 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.746407    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 09:07:47.750894 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.750875    2558 fs.go:135] Filesystem UUIDs: map[2ec49ecc-4f08-464e-bcba-eed43aaeed25:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b11fa79e-2297-4b20-91b4-2fdf58b40da9:/dev/nvme0n1p4]
Aug 04 09:07:47.750953 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.750895    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 09:07:47.756858 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.756825    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 09:07:47.758475 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.758353    2558 manager.go:216] Machine: {Timestamp:2026-08-04 09:07:47.756820861 +0000 UTC m=+0.376736971 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097209 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26bea8ed1a2969d881fb6706abbf2f SystemUUID:ec26bea8-ed1a-2969-d881-fb6706abbf2f BootID:0ad0661b-3aea-49ac-b881-3c3ef2f99e0d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a7:ef:1f:39:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a7:ef:1f:39:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:5d:ea:3f:66:c4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 09:07:47.758475 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.758475    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 09:07:47.758569 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.758551    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 09:07:47.762044 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.762022    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 09:07:47.762196 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.762047    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-106.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 09:07:47.763551 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.763541    2558 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 09:07:47.763590 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.763555    2558 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 09:07:47.763590 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.763562    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 09:07:47.764232 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.764222    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 09:07:47.765484 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.765474    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 09:07:47.770766 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.770754    2558 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 09:07:47.770898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.770889    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 09:07:47.770928 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.770920    2558 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 09:07:47.770956 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.770929    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 09:07:47.770956 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.770943    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 09:07:47.770956 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.770950    2558 kubelet.go:398] "Adding apiserver pod source"
Aug 04 09:07:47.771062 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.770959    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 09:07:47.771298 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.771287    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 09:07:47.771343 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.771307    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 09:07:47.774286 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.774269    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 09:07:47.775904 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.775891    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 09:07:47.775953 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.775921    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 09:07:47.777373 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777350    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 09:07:47.777373 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777374    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 09:07:47.777373 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777381    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777386    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777395    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777399    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777407    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777412    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777418    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777423    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777438    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 09:07:47.777541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.777444    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 09:07:47.778359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.778348    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 09:07:47.778359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.778359    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 09:07:47.781010 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.780992    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zqr7b"
Aug 04 09:07:47.781880 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.781866    2558 server.go:1268] "Started kubelet"
Aug 04 09:07:47.782010 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.781951    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 09:07:47.786546 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.786510    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 09:07:47.786646 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.786598    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 09:07:47.786927 ip-10-0-143-106 systemd[1]: Started Kubernetes Kubelet.
Aug 04 09:07:47.787535 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.787118    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 09:07:47.787535 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.787388    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-106.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 09:07:47.787729 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.787530    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 09:07:47.788072 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.788048    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 09:07:47.789182 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.789144    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zqr7b"
Aug 04 09:07:47.789294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.789276    2558 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 09:07:47.789491 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.789461    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 09:07:47.793728 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.792666    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-106.ec2.internal.18c88efc97c901c2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-106.ec2.internal,UID:ip-10-0-143-106.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-106.ec2.internal,},FirstTimestamp:2026-08-04 09:07:47.78183725 +0000 UTC m=+0.401753359,LastTimestamp:2026-08-04 09:07:47.78183725 +0000 UTC m=+0.401753359,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-106.ec2.internal,}"
Aug 04 09:07:47.796929 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.796911    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 09:07:47.796929 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.796920    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 09:07:47.797069 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.797008    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:47.797683 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.797663    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 09:07:47.797683 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.797665    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 09:07:47.797836 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.797706    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 09:07:47.797886 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.797836    2558 factory.go:55] Registering systemd factory
Aug 04 09:07:47.797886 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.797846    2558 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 09:07:47.797886 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.797855    2558 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 09:07:47.797886 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.797850    2558 factory.go:223] Registration of the systemd container factory successfully
Aug 04 09:07:47.798085 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.798072    2558 factory.go:153] Registering CRI-O factory
Aug 04 09:07:47.798137 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.798089    2558 factory.go:223] Registration of the crio container factory successfully
Aug 04 09:07:47.798187 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.798137    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 09:07:47.798187 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.798159    2558 factory.go:103] Registering Raw factory
Aug 04 09:07:47.798187 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.798173    2558 manager.go:1196] Started watching for new ooms in manager
Aug 04 09:07:47.798630 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.798614    2558 manager.go:349] Starting recovery of all containers
Aug 04 09:07:47.798974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.798959    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:47.801392 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.801371    2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-106.ec2.internal\" not found" node="ip-10-0-143-106.ec2.internal"
Aug 04 09:07:47.809647 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.809501    2558 manager.go:354] Recovery completed
Aug 04 09:07:47.814111 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.814097    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 09:07:47.815807 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.815791    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 09:07:47.815877 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.815823    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 09:07:47.815877 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.815841    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientPID"
Aug 04 09:07:47.816401 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.816384    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 09:07:47.816491 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.816407    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 09:07:47.816491 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.816425    2558 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 09:07:47.816491 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.816405    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 09:07:47.817313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.817270    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 09:07:47.817313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.817305    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 09:07:47.817454 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.817321    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientPID"
Aug 04 09:07:47.818584 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.818573    2558 policy_none.go:49] "None policy: Start"
Aug 04 09:07:47.818623 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.818597    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 09:07:47.818623 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.818608    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 09:07:47.820617 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.820608    2558 policy_none.go:47] "Start"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.863306    2558 manager.go:337] "Starting Device Plugin manager"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.863355    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.863367    2558 server.go:87] "Starting device plugin registration server"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.863596    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.863608    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.863703    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.863795    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.863804    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.864332    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 09:07:47.874977 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.864373    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:47.925747 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.925721    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 09:07:47.926999 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.926981    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 09:07:47.927081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.927020    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 09:07:47.927081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.927043    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 09:07:47.927185 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.927081    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 09:07:47.929366 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.929345    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:47.964161 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.964113    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 09:07:47.964921 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.964909    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 09:07:47.964974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.964935    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 09:07:47.964974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.964946    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientPID"
Aug 04 09:07:47.964974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.964968    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-106.ec2.internal"
Aug 04 09:07:47.971445 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:47.971432    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-106.ec2.internal"
Aug 04 09:07:47.971496 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.971450    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-106.ec2.internal\": node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:47.985413 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:47.985394    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:48.027709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.027688    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"]
Aug 04 09:07:48.027764 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.027742    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 09:07:48.028495 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.028481    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 09:07:48.028546 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.028508    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 09:07:48.028546 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.028521    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientPID"
Aug 04 09:07:48.030923 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.030909    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 09:07:48.031034 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031021    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.031071 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031051    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 09:07:48.031713 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031701    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 09:07:48.031713 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031710    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 09:07:48.031870 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031719    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 09:07:48.031870 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031727    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 09:07:48.031870 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031730    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientPID"
Aug 04 09:07:48.031870 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.031741    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientPID"
Aug 04 09:07:48.033973 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.033960    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.034019 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.033983    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 09:07:48.034658 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.034644    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 09:07:48.034714 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.034671    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 09:07:48.034714 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.034680    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeHasSufficientPID"
Aug 04 09:07:48.051308 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.051291    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-106.ec2.internal\" not found" node="ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.055544 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.055528    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-106.ec2.internal\" not found" node="ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.085821 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.085804    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:48.099178 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.099158    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc8da4866fc9d714207467948c0e50e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal\" (UID: \"bc8da4866fc9d714207467948c0e50e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.099233 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.099188    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc8da4866fc9d714207467948c0e50e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal\" (UID: \"bc8da4866fc9d714207467948c0e50e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.099233 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.099205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/862d5715eae1ca6077f67d5d3e2e5c27-config\") pod \"kube-apiserver-proxy-ip-10-0-143-106.ec2.internal\" (UID: \"862d5715eae1ca6077f67d5d3e2e5c27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.186551 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.186519    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:48.199885 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.199865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc8da4866fc9d714207467948c0e50e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal\" (UID: \"bc8da4866fc9d714207467948c0e50e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.199971 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.199896    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/862d5715eae1ca6077f67d5d3e2e5c27-config\") pod \"kube-apiserver-proxy-ip-10-0-143-106.ec2.internal\" (UID: \"862d5715eae1ca6077f67d5d3e2e5c27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.200014 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.199966    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc8da4866fc9d714207467948c0e50e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal\" (UID: \"bc8da4866fc9d714207467948c0e50e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.200014 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.199983    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc8da4866fc9d714207467948c0e50e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal\" (UID: \"bc8da4866fc9d714207467948c0e50e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.200014 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.199944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/862d5715eae1ca6077f67d5d3e2e5c27-config\") pod \"kube-apiserver-proxy-ip-10-0-143-106.ec2.internal\" (UID: \"862d5715eae1ca6077f67d5d3e2e5c27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.200119 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.200015    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc8da4866fc9d714207467948c0e50e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal\" (UID: \"bc8da4866fc9d714207467948c0e50e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.287299 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.287279    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:48.353830 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.353798    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.360144 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.359051    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.387981 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.387953    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:48.488452 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.488424    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:48.589007 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.588948    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-106.ec2.internal\" not found"
Aug 04 09:07:48.653453 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.653433    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:48.687721 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.687699    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:48.698162 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.698139    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.710501 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.710480    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 09:07:48.711317 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.711301    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"
Aug 04 09:07:48.717561 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.717546    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 09:07:48.726692 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.726677    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 09:07:48.726873 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.726843    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 09:07:48.726873 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.726846    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 09:07:48.726996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.726846    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 09:07:48.726996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.726851    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 09:07:48.772009 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.771988    2558 apiserver.go:52] "Watching apiserver"
Aug 04 09:07:48.787002 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.786980    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 09:07:48.788410 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.788386    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal","openshift-multus/multus-additional-cni-plugins-kpmrg","openshift-network-diagnostics/network-check-target-l9zs2","openshift-ovn-kubernetes/ovnkube-node-jsvrt","kube-system/global-pull-secret-syncer-kskw5","kube-system/konnectivity-agent-42hvz","openshift-cluster-node-tuning-operator/tuned-ngsvn","openshift-dns/node-resolver-dsh54","openshift-image-registry/node-ca-pd75f","openshift-multus/multus-9tbp6","openshift-multus/network-metrics-daemon-c852f","openshift-network-operator/iptables-alerter-bhl94","kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal"]
Aug 04 09:07:48.791092 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.791071    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:48.795107 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.793295    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 09:07:48.795107 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.794140    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 09:02:49 +0000 UTC" deadline="2028-02-28 10:23:45.447513348 +0000 UTC"
Aug 04 09:07:48.795107 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.794208    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13753h15m56.653310072s"
Aug 04 09:07:48.795107 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.794316    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6klgn\""
Aug 04 09:07:48.795107 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.794316    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 09:07:48.795394 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.795335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.797144 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.797127    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 09:07:48.797324 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.797309    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 09:07:48.797397 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.797362    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bvcfm\""
Aug 04 09:07:48.797463 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.797445    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 09:07:48.797519 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.797454    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 09:07:48.798231 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.798215    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.800055 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.800038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:48.800143 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.800117    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:07:48.800208 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.800191    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gzj64\""
Aug 04 09:07:48.800371 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.800356    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 09:07:48.800436 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.800423    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 09:07:48.800468 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.800439    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 09:07:48.800540 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.800513    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 09:07:48.800718 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.800706    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 09:07:48.802489 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.802475    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.802890 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.802871    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b76a8da3-d1b6-472d-bdea-aea40ac4f05d-agent-certs\") pod \"konnectivity-agent-42hvz\" (UID: \"b76a8da3-d1b6-472d-bdea-aea40ac4f05d\") " pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:48.802966 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.802896    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-cnibin\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.802966 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.802915    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-os-release\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.802966 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.802932    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-host-slash\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.803091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.802971    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.803091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803000    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.803091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803017    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpbsm\" (UniqueName: \"kubernetes.io/projected/a02136e4-8849-45d0-85db-46aac84612e9-kube-api-access-cpbsm\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.803091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803037    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:48.803091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803054    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.803294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803093    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b76a8da3-d1b6-472d-bdea-aea40ac4f05d-konnectivity-ca\") pod \"konnectivity-agent-42hvz\" (UID: \"b76a8da3-d1b6-472d-bdea-aea40ac4f05d\") " pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:48.803294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803123    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-system-cni-dir\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.803294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803151    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-cni-binary-copy\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.803294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803178    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-iptables-alerter-script\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.803294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.803203    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw8l7\" (UniqueName: \"kubernetes.io/projected/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-kube-api-access-fw8l7\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.804516 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.804500    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 09:07:48.804516 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.804522    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 09:07:48.804675 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.804628    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 09:07:48.804770 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.804754    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:48.804868 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.804850    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wnz2q\""
Aug 04 09:07:48.804921 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.804850    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:07:48.804975 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.804934    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 09:07:48.805074 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.805047    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 09:07:48.805149 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.805134    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 09:07:48.807141 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.807125    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:48.809302 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.809286    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.809802 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.809761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ttsfc\""
Aug 04 09:07:48.809903 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.809839    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 09:07:48.810364 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.810348    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 09:07:48.810455 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.810397    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 09:07:48.811313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.811297    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 09:07:48.811440 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.811423    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tmmwr\""
Aug 04 09:07:48.811485 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.811422    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 09:07:48.811804 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.811771    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.811946 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.811931    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 09:07:48.813835 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.813819    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 09:07:48.813931 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.813921    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 09:07:48.814000 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.813983    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6f7sq\""
Aug 04 09:07:48.814222 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.814208    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:48.816199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.816179    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 09:07:48.816379 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.816363    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 09:07:48.816458 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.816390    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m88qh\""
Aug 04 09:07:48.816458 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.816412    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 09:07:48.817097 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.817084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.818970 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.818945    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 09:07:48.819047 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.818995    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7ncv5\""
Aug 04 09:07:48.819410 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.819395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:48.819484 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.819454    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:07:48.833763 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.833740    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wpbh"
Aug 04 09:07:48.841843 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.841826    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wpbh"
Aug 04 09:07:48.877058 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:48.877037    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8da4866fc9d714207467948c0e50e4.slice/crio-97da61b8eb582391ffaa8cca1ef4878713258117d190bb45921a14f10ba15c6c WatchSource:0}: Error finding container 97da61b8eb582391ffaa8cca1ef4878713258117d190bb45921a14f10ba15c6c: Status 404 returned error can't find the container with id 97da61b8eb582391ffaa8cca1ef4878713258117d190bb45921a14f10ba15c6c
Aug 04 09:07:48.877436 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:48.877424    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod862d5715eae1ca6077f67d5d3e2e5c27.slice/crio-b778498101fe73deb2a6cd4b97d56e23be038201aef41a1e2c8d5aeca5f6bd8b WatchSource:0}: Error finding container b778498101fe73deb2a6cd4b97d56e23be038201aef41a1e2c8d5aeca5f6bd8b: Status 404 returned error can't find the container with id b778498101fe73deb2a6cd4b97d56e23be038201aef41a1e2c8d5aeca5f6bd8b
Aug 04 09:07:48.882657 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.882641    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 09:07:48.898389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.898367    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 09:07:48.903338 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903320    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-cnibin\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.903433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcac14e8-3c2b-4665-a56e-7fc16b48043f-tmp-dir\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:48.903433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd318799-d073-47ef-8d05-75070c90076e-etc-tuned\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.903433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-system-cni-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.903433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903422    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovnkube-config\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.903639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-cnibin\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.903639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903466    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.903639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903497    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-kubelet\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.903639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-slash\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.903639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.903639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-node-log\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.903639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-env-overrides\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b76a8da3-d1b6-472d-bdea-aea40ac4f05d-konnectivity-ca\") pod \"konnectivity-agent-42hvz\" (UID: \"b76a8da3-d1b6-472d-bdea-aea40ac4f05d\") " pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903770    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-etc-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903821    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-cni-netd\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903862    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovn-node-metrics-cert\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.903946    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw8l7\" (UniqueName: \"kubernetes.io/projected/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-kube-api-access-fw8l7\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904018    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpn8z\" (UniqueName: \"kubernetes.io/projected/bcac14e8-3c2b-4665-a56e-7fc16b48043f-kube-api-access-xpn8z\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904030    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904047    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2ch4\" (UniqueName: \"kubernetes.io/projected/dd318799-d073-47ef-8d05-75070c90076e-kube-api-access-r2ch4\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.904081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904068    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-os-release\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904093    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frf7k\" (UniqueName: \"kubernetes.io/projected/5596782e-d881-4c0a-9d59-4bcacc8ba154-kube-api-access-frf7k\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904122    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-sys-fs\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b76a8da3-d1b6-472d-bdea-aea40ac4f05d-agent-certs\") pod \"konnectivity-agent-42hvz\" (UID: \"b76a8da3-d1b6-472d-bdea-aea40ac4f05d\") " pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904156    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b76a8da3-d1b6-472d-bdea-aea40ac4f05d-konnectivity-ca\") pod \"konnectivity-agent-42hvz\" (UID: \"b76a8da3-d1b6-472d-bdea-aea40ac4f05d\") " pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904177    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-dbus\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-var-lib-kubelet\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904227    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-log-socket\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904297    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-kubelet-config\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904322    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-sys\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ccbe261-c0cc-4deb-8575-27d331f415d3-host\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904370    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-k8s-cni-cncf-io\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904398    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-cni-multus\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904421    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-run-netns\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.904460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904447    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-ovn\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904467    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904476    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-cni-binary-copy\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904499    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-cni-binary-copy\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-hostroot\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904544    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-etc-kubernetes\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904568    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-run-ovn-kubernetes\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904604    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzfcs\" (UniqueName: \"kubernetes.io/projected/4ccbe261-c0cc-4deb-8575-27d331f415d3-kube-api-access-dzfcs\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904660    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-kubernetes\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904683    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-os-release\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904742    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-cni-bin\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-os-release\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904769    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpbsm\" (UniqueName: \"kubernetes.io/projected/a02136e4-8849-45d0-85db-46aac84612e9-kube-api-access-cpbsm\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904842    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-modprobe-d\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904858    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysctl-conf\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.905089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-host\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904885    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-cnibin\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904905    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-cni-binary-copy\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904914    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr4rw\" (UniqueName: \"kubernetes.io/projected/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-kube-api-access-jr4rw\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-systemd\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904959    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-systemd-units\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.904981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905018    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-iptables-alerter-script\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905047    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-run\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905061    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-netns\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905078    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-etc-selinux\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905095    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysctl-d\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905109    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcac14e8-3c2b-4665-a56e-7fc16b48043f-hosts-file\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905122    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-cni-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905139    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-multus-certs\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovnkube-script-lib\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905188    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pvpf\" (UniqueName: \"kubernetes.io/projected/7d08c72f-7659-492c-8f62-e6455f3b6b84-kube-api-access-7pvpf\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.905996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-daemon-config\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-host-slash\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905289    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-host-slash\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905338    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysconfig\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905362    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-systemd\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905381    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905397    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnjd2\" (UniqueName: \"kubernetes.io/projected/65b7539a-5c74-4647-9e2e-87e2a3961cd3-kube-api-access-rnjd2\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905422    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-system-cni-dir\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-socket-dir-parent\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-system-cni-dir\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905490    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a02136e4-8849-45d0-85db-46aac84612e9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905490    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-kubelet\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905522    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-var-lib-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-registration-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.906737 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905553    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-device-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905574    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-lib-modules\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905579    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-iptables-alerter-script\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905602    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd318799-d073-47ef-8d05-75070c90076e-tmp\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905643    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ccbe261-c0cc-4deb-8575-27d331f415d3-serviceca\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905678    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-cni-bin\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905701    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-conf-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905723    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-socket-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:48.907480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.905984    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a02136e4-8849-45d0-85db-46aac84612e9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.908199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.908183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b76a8da3-d1b6-472d-bdea-aea40ac4f05d-agent-certs\") pod \"konnectivity-agent-42hvz\" (UID: \"b76a8da3-d1b6-472d-bdea-aea40ac4f05d\") " pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:48.910905 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.910889    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 09:07:48.910987 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.910915    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 09:07:48.910987 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.910927    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vckwh for pod openshift-network-diagnostics/network-check-target-l9zs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:48.910987 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:48.910983    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh podName:95d2561a-6816-4955-b1e7-e8b10e974687 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:49.410964468 +0000 UTC m=+2.030880583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vckwh" (UniqueName: "kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh") pod "network-check-target-l9zs2" (UID: "95d2561a-6816-4955-b1e7-e8b10e974687") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:48.913303 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.913283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw8l7\" (UniqueName: \"kubernetes.io/projected/3ac224d3-8dd7-4196-9efb-0dc2a00275f9-kube-api-access-fw8l7\") pod \"iptables-alerter-bhl94\" (UID: \"3ac224d3-8dd7-4196-9efb-0dc2a00275f9\") " pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:48.913426 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.913409    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpbsm\" (UniqueName: \"kubernetes.io/projected/a02136e4-8849-45d0-85db-46aac84612e9-kube-api-access-cpbsm\") pod \"multus-additional-cni-plugins-kpmrg\" (UID: \"a02136e4-8849-45d0-85db-46aac84612e9\") " pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:48.932944 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.932903    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal" event={"ID":"bc8da4866fc9d714207467948c0e50e4","Type":"ContainerStarted","Data":"97da61b8eb582391ffaa8cca1ef4878713258117d190bb45921a14f10ba15c6c"}
Aug 04 09:07:48.933847 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:48.933825    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal" event={"ID":"862d5715eae1ca6077f67d5d3e2e5c27","Type":"ContainerStarted","Data":"b778498101fe73deb2a6cd4b97d56e23be038201aef41a1e2c8d5aeca5f6bd8b"}
Aug 04 09:07:49.006947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.006925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-kubelet\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.006947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.006954    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-slash\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.006970    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.006990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-node-log\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-env-overrides\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-slash\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007034    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-etc-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007044    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-kubelet\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007054    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-cni-netd\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007064    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-node-log\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007062    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007084    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-cni-netd\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovn-node-metrics-cert\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpn8z\" (UniqueName: \"kubernetes.io/projected/bcac14e8-3c2b-4665-a56e-7fc16b48043f-kube-api-access-xpn8z\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007117    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-etc-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007128    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2ch4\" (UniqueName: \"kubernetes.io/projected/dd318799-d073-47ef-8d05-75070c90076e-kube-api-access-r2ch4\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-os-release\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007203    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frf7k\" (UniqueName: \"kubernetes.io/projected/5596782e-d881-4c0a-9d59-4bcacc8ba154-kube-api-access-frf7k\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-sys-fs\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007252    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-dbus\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007257    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-os-release\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-var-lib-kubelet\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007300    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-log-socket\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007315    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-sys-fs\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-kubelet-config\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007359    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-var-lib-kubelet\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007360    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-sys\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ccbe261-c0cc-4deb-8575-27d331f415d3-host\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007401    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-sys\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007403    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-log-socket\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007421    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-k8s-cni-cncf-io\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.007669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007443    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ccbe261-c0cc-4deb-8575-27d331f415d3-host\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-dbus\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-cni-multus\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007465    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-k8s-cni-cncf-io\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007478    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-run-netns\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007456    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-kubelet-config\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-cni-multus\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007491    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-env-overrides\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007508    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-run-netns\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-ovn\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007538    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-cni-binary-copy\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007542    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-ovn\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-hostroot\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-etc-kubernetes\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-run-ovn-kubernetes\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-hostroot\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007633    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzfcs\" (UniqueName: \"kubernetes.io/projected/4ccbe261-c0cc-4deb-8575-27d331f415d3-kube-api-access-dzfcs\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007673    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-etc-kubernetes\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.008356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007679    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-run-ovn-kubernetes\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007682    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007709    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-kubernetes\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007738    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-kubernetes\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007742    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:49.007843    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:49.007900    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret podName:e4df6d71-b2d6-4b8d-af54-36717c5c4da5 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:49.507886732 +0000 UTC m=+2.127802827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret") pod "global-pull-secret-syncer-kskw5" (UID: "e4df6d71-b2d6-4b8d-af54-36717c5c4da5") : object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007869    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-cni-bin\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-modprobe-d\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysctl-conf\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.007945    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-cni-bin\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-host\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-cnibin\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008071    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr4rw\" (UniqueName: \"kubernetes.io/projected/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-kube-api-access-jr4rw\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-systemd\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-cni-binary-copy\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.009167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008141    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-cnibin\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008134    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysctl-conf\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008158    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-systemd-units\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-systemd\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008173    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-host\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008103    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-systemd-units\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008198    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-modprobe-d\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-run\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008235    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-netns\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008254    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-etc-selinux\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008279    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-run\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:49.008334    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-netns\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:49.008374    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs podName:65b7539a-5c74-4647-9e2e-87e2a3961cd3 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:49.508359473 +0000 UTC m=+2.128275587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs") pod "network-metrics-daemon-c852f" (UID: "65b7539a-5c74-4647-9e2e-87e2a3961cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysctl-d\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008401    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcac14e8-3c2b-4665-a56e-7fc16b48043f-hosts-file\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-cni-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.009853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-multus-certs\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008448    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-etc-selinux\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008458    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovnkube-script-lib\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcac14e8-3c2b-4665-a56e-7fc16b48043f-hosts-file\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008494    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-cni-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008493    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysctl-d\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008512    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-run-multus-certs\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008517    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pvpf\" (UniqueName: \"kubernetes.io/projected/7d08c72f-7659-492c-8f62-e6455f3b6b84-kube-api-access-7pvpf\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008568    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-daemon-config\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysconfig\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008619    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-systemd\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnjd2\" (UniqueName: \"kubernetes.io/projected/65b7539a-5c74-4647-9e2e-87e2a3961cd3-kube-api-access-rnjd2\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-socket-dir-parent\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008711    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-run-systemd\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-etc-sysconfig\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010456 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-kubelet\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008765    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-host-kubelet\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-var-lib-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008819    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-registration-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008828    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5596782e-d881-4c0a-9d59-4bcacc8ba154-var-lib-openvswitch\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-device-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-socket-dir-parent\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-lib-modules\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd318799-d073-47ef-8d05-75070c90076e-tmp\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ccbe261-c0cc-4deb-8575-27d331f415d3-serviceca\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008968    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-cni-bin\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008982    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd318799-d073-47ef-8d05-75070c90076e-lib-modules\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovnkube-script-lib\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008994    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-conf-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009021    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-socket-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009027    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-daemon-config\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.008969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-registration-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.010945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-host-var-lib-cni-bin\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009030    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-device-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcac14e8-3c2b-4665-a56e-7fc16b48043f-tmp-dir\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009218    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d08c72f-7659-492c-8f62-e6455f3b6b84-socket-dir\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009222    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd318799-d073-47ef-8d05-75070c90076e-etc-tuned\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009259    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-multus-conf-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-system-cni-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovnkube-config\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009362    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcac14e8-3c2b-4665-a56e-7fc16b48043f-tmp-dir\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009431    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-system-cni-dir\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009447    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ccbe261-c0cc-4deb-8575-27d331f415d3-serviceca\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.009746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovn-node-metrics-cert\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.010202    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5596782e-d881-4c0a-9d59-4bcacc8ba154-ovnkube-config\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.011165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd318799-d073-47ef-8d05-75070c90076e-tmp\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.011510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.011459    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dd318799-d073-47ef-8d05-75070c90076e-etc-tuned\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.020222 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.020193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr4rw\" (UniqueName: \"kubernetes.io/projected/6363ba3c-b1f8-454e-bf96-3e3ffc89c7af-kube-api-access-jr4rw\") pod \"multus-9tbp6\" (UID: \"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af\") " pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:49.020222 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.020216    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzfcs\" (UniqueName: \"kubernetes.io/projected/4ccbe261-c0cc-4deb-8575-27d331f415d3-kube-api-access-dzfcs\") pod \"node-ca-pd75f\" (UID: \"4ccbe261-c0cc-4deb-8575-27d331f415d3\") " pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:49.020409 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.020391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2ch4\" (UniqueName: \"kubernetes.io/projected/dd318799-d073-47ef-8d05-75070c90076e-kube-api-access-r2ch4\") pod \"tuned-ngsvn\" (UID: \"dd318799-d073-47ef-8d05-75070c90076e\") " pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:49.020449 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.020416    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pvpf\" (UniqueName: \"kubernetes.io/projected/7d08c72f-7659-492c-8f62-e6455f3b6b84-kube-api-access-7pvpf\") pod \"aws-ebs-csi-driver-node-ht6pw\" (UID: \"7d08c72f-7659-492c-8f62-e6455f3b6b84\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:49.020581 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.020567    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpn8z\" (UniqueName: \"kubernetes.io/projected/bcac14e8-3c2b-4665-a56e-7fc16b48043f-kube-api-access-xpn8z\") pod \"node-resolver-dsh54\" (UID: \"bcac14e8-3c2b-4665-a56e-7fc16b48043f\") " pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:49.020817 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.020803    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frf7k\" (UniqueName: \"kubernetes.io/projected/5596782e-d881-4c0a-9d59-4bcacc8ba154-kube-api-access-frf7k\") pod \"ovnkube-node-jsvrt\" (UID: \"5596782e-d881-4c0a-9d59-4bcacc8ba154\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.020898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.020883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnjd2\" (UniqueName: \"kubernetes.io/projected/65b7539a-5c74-4647-9e2e-87e2a3961cd3-kube-api-access-rnjd2\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:49.120943 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.120884    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:07:49.131604 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:49.131585    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76a8da3_d1b6_472d_bdea_aea40ac4f05d.slice/crio-34c179e8a281bf8dcb57b0e4a5f8080469bd363c74d33acf96cc3a0580d59a3b WatchSource:0}: Error finding container 34c179e8a281bf8dcb57b0e4a5f8080469bd363c74d33acf96cc3a0580d59a3b: Status 404 returned error can't find the container with id 34c179e8a281bf8dcb57b0e4a5f8080469bd363c74d33acf96cc3a0580d59a3b
Aug 04 09:07:49.138292 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.138277    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bhl94"
Aug 04 09:07:49.146062 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:49.146040    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac224d3_8dd7_4196_9efb_0dc2a00275f9.slice/crio-f1989083f2aedb8007e496dfaea4d99ac39eb7ad5c2652ffadffbc1adf8b70fb WatchSource:0}: Error finding container f1989083f2aedb8007e496dfaea4d99ac39eb7ad5c2652ffadffbc1adf8b70fb: Status 404 returned error can't find the container with id f1989083f2aedb8007e496dfaea4d99ac39eb7ad5c2652ffadffbc1adf8b70fb
Aug 04 09:07:49.159505 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.159487    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kpmrg"
Aug 04 09:07:49.163293 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:49.163272    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:07:49.168102 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:49.168070    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02136e4_8849_45d0_85db_46aac84612e9.slice/crio-a9f895ad4a5286f76e59b74723a18223d255bf57b9b0558c412ccba8a02504e8 WatchSource:0}: Error finding container a9f895ad4a5286f76e59b74723a18223d255bf57b9b0558c412ccba8a02504e8: Status 404 returned error can't find the container with id a9f895ad4a5286f76e59b74723a18223d255bf57b9b0558c412ccba8a02504e8
Aug 04 09:07:49.172928 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:49.172904    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5596782e_d881_4c0a_9d59_4bcacc8ba154.slice/crio-66b36ce9a448215e53e2c218c88cc0676072a2eda4b00a2c3dab697c08737d49 WatchSource:0}: Error finding container 66b36ce9a448215e53e2c218c88cc0676072a2eda4b00a2c3dab697c08737d49: Status 404 returned error can't find the container with id 66b36ce9a448215e53e2c218c88cc0676072a2eda4b00a2c3dab697c08737d49
Aug 04 09:07:50.546909 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.546892    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dsh54"
Aug 04 09:07:50.552589 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.552573    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw"
Aug 04 09:07:50.555551 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:50.555535    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcac14e8_3c2b_4665_a56e_7fc16b48043f.slice/crio-01ade7014bcc144169e71e8977f5015325a117cbcb4971f7f2f9c5b4e7803410 WatchSource:0}: Error finding container 01ade7014bcc144169e71e8977f5015325a117cbcb4971f7f2f9c5b4e7803410: Status 404 returned error can't find the container with id 01ade7014bcc144169e71e8977f5015325a117cbcb4971f7f2f9c5b4e7803410
Aug 04 09:07:50.557281 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.557267    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ngsvn"
Aug 04 09:07:50.560344 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:50.560323    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d08c72f_7659_492c_8f62_e6455f3b6b84.slice/crio-feaa31c6b80ec64f84d9e4fa9de0f110d6aa31e5803b7b3c12edbd6da9ec5762 WatchSource:0}: Error finding container feaa31c6b80ec64f84d9e4fa9de0f110d6aa31e5803b7b3c12edbd6da9ec5762: Status 404 returned error can't find the container with id feaa31c6b80ec64f84d9e4fa9de0f110d6aa31e5803b7b3c12edbd6da9ec5762
Aug 04 09:07:50.562846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.562827    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pd75f"
Aug 04 09:07:50.565387 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:50.565371    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd318799_d073_47ef_8d05_75070c90076e.slice/crio-788c07dc65b2f3c236e297eaf30dafcc34371299a6dc968940d31c89b1a70783 WatchSource:0}: Error finding container 788c07dc65b2f3c236e297eaf30dafcc34371299a6dc968940d31c89b1a70783: Status 404 returned error can't find the container with id 788c07dc65b2f3c236e297eaf30dafcc34371299a6dc968940d31c89b1a70783
Aug 04 09:07:50.568361 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.568346    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9tbp6"
Aug 04 09:07:50.572129 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:50.572108    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ccbe261_c0cc_4deb_8575_27d331f415d3.slice/crio-bbae2fdbc75a682a6d4e1f3c2cdde57d5fdbb8d0295c5973ad5304017fc6a11c WatchSource:0}: Error finding container bbae2fdbc75a682a6d4e1f3c2cdde57d5fdbb8d0295c5973ad5304017fc6a11c: Status 404 returned error can't find the container with id bbae2fdbc75a682a6d4e1f3c2cdde57d5fdbb8d0295c5973ad5304017fc6a11c
Aug 04 09:07:50.577302 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:07:50.577286    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6363ba3c_b1f8_454e_bf96_3e3ffc89c7af.slice/crio-ea5c46515ff2b877d8b77693b5c14e507ce1b3424c2025fb01dff87ad7305172 WatchSource:0}: Error finding container ea5c46515ff2b877d8b77693b5c14e507ce1b3424c2025fb01dff87ad7305172: Status 404 returned error can't find the container with id ea5c46515ff2b877d8b77693b5c14e507ce1b3424c2025fb01dff87ad7305172
Aug 04 09:07:50.773825 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.773728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:50.773973 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.773924    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 09:07:50.773973 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.773948    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 09:07:50.773973 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.773961    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vckwh for pod openshift-network-diagnostics/network-check-target-l9zs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:50.774116 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.774032    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh podName:95d2561a-6816-4955-b1e7-e8b10e974687 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:51.774005098 +0000 UTC m=+3.031962144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vckwh" (UniqueName: "kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh") pod "network-check-target-l9zs2" (UID: "95d2561a-6816-4955-b1e7-e8b10e974687") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:50.874396 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.874364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:50.874562 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.874408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:50.874562 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.874485    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:50.874562 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.874534    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:50.874562 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.874553    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret podName:e4df6d71-b2d6-4b8d-af54-36717c5c4da5 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:51.874532722 +0000 UTC m=+3.132489771 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret") pod "global-pull-secret-syncer-kskw5" (UID: "e4df6d71-b2d6-4b8d-af54-36717c5c4da5") : object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:50.874811 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:50.874572    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs podName:65b7539a-5c74-4647-9e2e-87e2a3961cd3 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:51.874559883 +0000 UTC m=+3.132516928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs") pod "network-metrics-daemon-c852f" (UID: "65b7539a-5c74-4647-9e2e-87e2a3961cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:50.996278 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:50.996245    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:51.207529 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.207414    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:02:50 +0000 UTC" deadline="2028-05-03 13:53:20.037381514 +0000 UTC"
Aug 04 09:07:51.207529 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.207470    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15316h45m28.829914945s"
Aug 04 09:07:51.290483 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.289959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:51.290483 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.290109    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:07:51.309079 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.309042    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw" event={"ID":"7d08c72f-7659-492c-8f62-e6455f3b6b84","Type":"ContainerStarted","Data":"feaa31c6b80ec64f84d9e4fa9de0f110d6aa31e5803b7b3c12edbd6da9ec5762"}
Aug 04 09:07:51.323884 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.323853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9tbp6" event={"ID":"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af","Type":"ContainerStarted","Data":"ea5c46515ff2b877d8b77693b5c14e507ce1b3424c2025fb01dff87ad7305172"}
Aug 04 09:07:51.326426 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.326397    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pd75f" event={"ID":"4ccbe261-c0cc-4deb-8575-27d331f415d3","Type":"ContainerStarted","Data":"bbae2fdbc75a682a6d4e1f3c2cdde57d5fdbb8d0295c5973ad5304017fc6a11c"}
Aug 04 09:07:51.328181 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.328155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ngsvn" event={"ID":"dd318799-d073-47ef-8d05-75070c90076e","Type":"ContainerStarted","Data":"788c07dc65b2f3c236e297eaf30dafcc34371299a6dc968940d31c89b1a70783"}
Aug 04 09:07:51.338345 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.338309    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dsh54" event={"ID":"bcac14e8-3c2b-4665-a56e-7fc16b48043f","Type":"ContainerStarted","Data":"01ade7014bcc144169e71e8977f5015325a117cbcb4971f7f2f9c5b4e7803410"}
Aug 04 09:07:51.341890 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.341859    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"66b36ce9a448215e53e2c218c88cc0676072a2eda4b00a2c3dab697c08737d49"}
Aug 04 09:07:51.348853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.348824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerStarted","Data":"a9f895ad4a5286f76e59b74723a18223d255bf57b9b0558c412ccba8a02504e8"}
Aug 04 09:07:51.349293 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.349274    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:51.353278 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.353249    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bhl94" event={"ID":"3ac224d3-8dd7-4196-9efb-0dc2a00275f9","Type":"ContainerStarted","Data":"f1989083f2aedb8007e496dfaea4d99ac39eb7ad5c2652ffadffbc1adf8b70fb"}
Aug 04 09:07:51.357562 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.357533    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-42hvz" event={"ID":"b76a8da3-d1b6-472d-bdea-aea40ac4f05d","Type":"ContainerStarted","Data":"34c179e8a281bf8dcb57b0e4a5f8080469bd363c74d33acf96cc3a0580d59a3b"}
Aug 04 09:07:51.763069 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.763041    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:51.781557 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.781522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:51.781697 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.781679    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 09:07:51.781792 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.781702    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 09:07:51.781792 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.781715    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vckwh for pod openshift-network-diagnostics/network-check-target-l9zs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:51.781910 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.781792    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh podName:95d2561a-6816-4955-b1e7-e8b10e974687 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:53.781772286 +0000 UTC m=+5.039729320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vckwh" (UniqueName: "kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh") pod "network-check-target-l9zs2" (UID: "95d2561a-6816-4955-b1e7-e8b10e974687") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:51.818406 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.818377    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 09:07:51.882922 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.882193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:51.882922 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:51.882240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:51.882922 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.882375    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:51.882922 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.882433    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs podName:65b7539a-5c74-4647-9e2e-87e2a3961cd3 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:53.882414436 +0000 UTC m=+5.140371480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs") pod "network-metrics-daemon-c852f" (UID: "65b7539a-5c74-4647-9e2e-87e2a3961cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:51.882922 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.882835    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:51.882922 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:51.882885    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret podName:e4df6d71-b2d6-4b8d-af54-36717c5c4da5 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:53.882869958 +0000 UTC m=+5.140826994 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret") pod "global-pull-secret-syncer-kskw5" (UID: "e4df6d71-b2d6-4b8d-af54-36717c5c4da5") : object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:52.208015 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:52.207970    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:02:50 +0000 UTC" deadline="2028-01-05 00:17:53.313607355 +0000 UTC"
Aug 04 09:07:52.208015 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:52.208008    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12447h10m1.105603892s"
Aug 04 09:07:52.289852 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:52.289822    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:52.290024 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:52.289950    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:07:52.290353 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:52.290331    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:52.290445 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:52.290427    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:07:53.291943 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:53.291890    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:53.292481 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.292032    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:07:53.799360 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:53.799308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:53.799560 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.799535    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 09:07:53.799627 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.799560    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 09:07:53.799627 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.799576    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vckwh for pod openshift-network-diagnostics/network-check-target-l9zs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:53.799725 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.799632    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh podName:95d2561a-6816-4955-b1e7-e8b10e974687 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:57.799614027 +0000 UTC m=+9.057571061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vckwh" (UniqueName: "kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh") pod "network-check-target-l9zs2" (UID: "95d2561a-6816-4955-b1e7-e8b10e974687") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:53.900772 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:53.900719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:53.900947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:53.900793    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:53.901012 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.900956    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:53.901068 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.901024    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs podName:65b7539a-5c74-4647-9e2e-87e2a3961cd3 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:57.90100382 +0000 UTC m=+9.158960853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs") pod "network-metrics-daemon-c852f" (UID: "65b7539a-5c74-4647-9e2e-87e2a3961cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:53.901430 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.901408    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:53.901517 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:53.901466    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret podName:e4df6d71-b2d6-4b8d-af54-36717c5c4da5 nodeName:}" failed. No retries permitted until 2026-08-04 09:07:57.901451468 +0000 UTC m=+9.159408507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret") pod "global-pull-secret-syncer-kskw5" (UID: "e4df6d71-b2d6-4b8d-af54-36717c5c4da5") : object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:54.290072 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:54.290033    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:54.290072 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:54.290075    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:54.290361 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:54.290177    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:07:54.290515 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:54.290488    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:07:55.289776 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:55.289728    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:55.290231 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:55.289884    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:07:56.289871 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:56.289835    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:56.290356 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:56.289966    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:07:56.290356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:56.290334    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:56.290482 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:56.290421    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:07:57.291379 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:57.290901    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:57.291379 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.291055    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:07:57.830920 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:57.830651    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:57.831061 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.830933    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 09:07:57.831061 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.830962    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 09:07:57.831061 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.830975    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vckwh for pod openshift-network-diagnostics/network-check-target-l9zs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:57.831061 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.831040    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh podName:95d2561a-6816-4955-b1e7-e8b10e974687 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:05.831020493 +0000 UTC m=+17.088977526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vckwh" (UniqueName: "kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh") pod "network-check-target-l9zs2" (UID: "95d2561a-6816-4955-b1e7-e8b10e974687") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:07:57.931493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:57.931453    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:57.931680 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:57.931512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:57.931680 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.931611    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:57.931680 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.931642    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:57.931872 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.931689    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret podName:e4df6d71-b2d6-4b8d-af54-36717c5c4da5 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:05.931665824 +0000 UTC m=+17.189622860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret") pod "global-pull-secret-syncer-kskw5" (UID: "e4df6d71-b2d6-4b8d-af54-36717c5c4da5") : object "kube-system"/"original-pull-secret" not registered
Aug 04 09:07:57.931872 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:57.931710    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs podName:65b7539a-5c74-4647-9e2e-87e2a3961cd3 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:05.931700002 +0000 UTC m=+17.189657033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs") pod "network-metrics-daemon-c852f" (UID: "65b7539a-5c74-4647-9e2e-87e2a3961cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:07:58.289896 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:58.289867    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:07:58.290068 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:58.289991    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:07:58.290340 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:58.290319    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:07:58.290440 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:58.290420    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:07:59.290291 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:07:59.290253    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:07:59.290771 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:07:59.290381    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:00.289837 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:00.289804    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:00.289837 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:00.289818    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:00.290055 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:00.289933    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:00.290366 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:00.290343    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:01.289418 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:01.289385    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:01.289628 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:01.289603    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:02.289622 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:02.289592    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:02.289953 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:02.289689    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:02.289953 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:02.289750    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:02.289953 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:02.289839    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:03.290018 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:03.289983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:03.290476 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:03.290140    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:04.290103 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:04.290065    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:04.290103 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:04.290113    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:04.290514 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:04.290216    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:04.290514 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:04.290361    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:05.289944 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:05.289914    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:05.290126 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.290040    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:05.889347 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:05.889312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:05.889548 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.889441    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 09:08:05.889548 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.889463    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 09:08:05.889548 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.889473    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vckwh for pod openshift-network-diagnostics/network-check-target-l9zs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:08:05.889548 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.889533    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh podName:95d2561a-6816-4955-b1e7-e8b10e974687 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:21.889515095 +0000 UTC m=+33.147472138 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vckwh" (UniqueName: "kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh") pod "network-check-target-l9zs2" (UID: "95d2561a-6816-4955-b1e7-e8b10e974687") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:08:05.989628 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:05.989589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:05.989628 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:05.989630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:05.989801 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.989755    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 09:08:05.989801 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.989763    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:08:05.989883 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.989811    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret podName:e4df6d71-b2d6-4b8d-af54-36717c5c4da5 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:21.9897965 +0000 UTC m=+33.247753532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret") pod "global-pull-secret-syncer-kskw5" (UID: "e4df6d71-b2d6-4b8d-af54-36717c5c4da5") : object "kube-system"/"original-pull-secret" not registered
Aug 04 09:08:05.989883 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:05.989826    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs podName:65b7539a-5c74-4647-9e2e-87e2a3961cd3 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:21.98981803 +0000 UTC m=+33.247775062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs") pod "network-metrics-daemon-c852f" (UID: "65b7539a-5c74-4647-9e2e-87e2a3961cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:08:06.290073 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:06.290006    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:06.290197 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:06.290005    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:06.290197 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:06.290146    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:06.290197 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:06.290182    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:07.289429 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:07.289399    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:07.289600 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:07.289554    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:08.290069 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:08.290037    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:08.290069 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:08.290067    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:08.290564 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:08.290165    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:08.290564 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:08.290322    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:09.290648 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:09.290613    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:09.291065 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:09.290756    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:10.290056 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:10.290021    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:10.290056 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:10.290057    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:10.290241 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:10.290137    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:10.290279 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:10.290250    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:11.289910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.289750    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:11.290497 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:11.289993    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:11.404941 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.404876    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9tbp6" event={"ID":"6363ba3c-b1f8-454e-bf96-3e3ffc89c7af","Type":"ContainerStarted","Data":"2c1ef54711caa3efe3a4ab415ea8f8dd5cae2a44dfa197e96b08014bf845f20d"}
Aug 04 09:08:11.406336 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.406314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ngsvn" event={"ID":"dd318799-d073-47ef-8d05-75070c90076e","Type":"ContainerStarted","Data":"48f2000e8e02b9259a8b198614626a00f33feb2ce030bde7a8bb61318a826053"}
Aug 04 09:08:11.408972 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.408944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"4384c7407d8fe9692d28bc80c39dee26bcf8960a6bc993e57082cf5a7f3972a6"}
Aug 04 09:08:11.409090 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.409018    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"fd1403f13f20fecc71583d26e7754d16caca003317e945b9a8f15292526c6155"}
Aug 04 09:08:11.409090 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.409069    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"c1c18740ab1d82b114b11c13b34d0d63cf20732e2c45679745b7383cd00d25d4"}
Aug 04 09:08:11.409188 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.409116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"e9143d1170c3294057a7c400723cdf16626c0e8e14529e1cda262387b8decc03"}
Aug 04 09:08:11.410306 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.410282    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal" event={"ID":"862d5715eae1ca6077f67d5d3e2e5c27","Type":"ContainerStarted","Data":"599f8e4131c9ba2401f52ee73056976e7f6f9705226b2c97c06e6583ee03aa01"}
Aug 04 09:08:11.424414 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.424370    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9tbp6" podStartSLOduration=2.364756642 podStartE2EDuration="22.424358529s" totalImagesPullingTime="20.059601887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:11.423874083 +0000 UTC m=+22.681831137" watchObservedRunningTime="2026-08-04 09:08:11.424358529 +0000 UTC m=+22.682315583"
Aug 04 09:08:11.441487 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.441116    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ngsvn" podStartSLOduration=2.371411186 podStartE2EDuration="22.441098193s" totalImagesPullingTime="20.069687007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:11.440645783 +0000 UTC m=+22.698602837" watchObservedRunningTime="2026-08-04 09:08:11.441098193 +0000 UTC m=+22.699055251"
Aug 04 09:08:11.459496 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:11.459450    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-106.ec2.internal" podStartSLOduration=21.459432036 podStartE2EDuration="21.459432036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:11.458797931 +0000 UTC m=+22.716755181" watchObservedRunningTime="2026-08-04 09:08:11.459432036 +0000 UTC m=+22.717389091"
Aug 04 09:08:12.290035 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.289819    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:12.290430 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.289877    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:12.290430 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:12.290065    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:12.290430 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:12.290142    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:12.415026 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.414605    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pd75f" event={"ID":"4ccbe261-c0cc-4deb-8575-27d331f415d3","Type":"ContainerStarted","Data":"3b114ed564a3177d71d4f879a568ad6ccfd9ec0f0b2ad7ee47d48b387bc9f4e4"}
Aug 04 09:08:12.416338 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.416308    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dsh54" event={"ID":"bcac14e8-3c2b-4665-a56e-7fc16b48043f","Type":"ContainerStarted","Data":"0c478eae243c248b4fced6f7c538e5ccfe840be13f7d4b4d52811de4f250b026"}
Aug 04 09:08:12.422687 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.422636    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"47d259e8ae92fff0bd1b889ddd3ac3b4ffc21d7e62687259a5c897553dc45702"}
Aug 04 09:08:12.422813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.422765    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"0b6dfb7f9dee8a5a724093cdb268f23be7cb8155c658bd7b93cf277c9e943ae7"}
Aug 04 09:08:12.424305 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.424279    2558 generic.go:356] "Generic (PLEG): container finished" podID="a02136e4-8849-45d0-85db-46aac84612e9" containerID="95c2e44de2c7f7a94cdda1399e7b8587dd072db3b1e5d1be44cfb474ed0526b5" exitCode=0
Aug 04 09:08:12.424408 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.424372    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerDied","Data":"95c2e44de2c7f7a94cdda1399e7b8587dd072db3b1e5d1be44cfb474ed0526b5"}
Aug 04 09:08:12.425909 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.425869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bhl94" event={"ID":"3ac224d3-8dd7-4196-9efb-0dc2a00275f9","Type":"ContainerStarted","Data":"b38b902b24da8d435017888e9fa05bd001746efd4204bc5bf8d7756ae50ca779"}
Aug 04 09:08:12.427792 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.427770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-42hvz" event={"ID":"b76a8da3-d1b6-472d-bdea-aea40ac4f05d","Type":"ContainerStarted","Data":"eb365e81152bfce18ea9c493e3f58afd22c86c64c86f3b794ec824200b9a5e4a"}
Aug 04 09:08:12.429359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.429340    2558 generic.go:356] "Generic (PLEG): container finished" podID="bc8da4866fc9d714207467948c0e50e4" containerID="bf0bd49ae92bb384d04de8682875c0169532e9c08718f72568e69190a620978a" exitCode=0
Aug 04 09:08:12.429446 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.429406    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal" event={"ID":"bc8da4866fc9d714207467948c0e50e4","Type":"ContainerDied","Data":"bf0bd49ae92bb384d04de8682875c0169532e9c08718f72568e69190a620978a"}
Aug 04 09:08:12.431792 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.430639    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pd75f" podStartSLOduration=3.370558402 podStartE2EDuration="23.430622401s" totalImagesPullingTime="20.060063999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:12.430302345 +0000 UTC m=+23.688259412" watchObservedRunningTime="2026-08-04 09:08:12.430622401 +0000 UTC m=+23.688579455"
Aug 04 09:08:12.431792 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.431377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw" event={"ID":"7d08c72f-7659-492c-8f62-e6455f3b6b84","Type":"ContainerStarted","Data":"e229b8ba0daf71561227125f95c053541cf4daa5264939ebfcbbace0781f153c"}
Aug 04 09:08:12.445077 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.445037    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-42hvz" podStartSLOduration=11.630549659 podStartE2EDuration="23.445022599s" totalImagesPullingTime="11.81447294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:12.444607081 +0000 UTC m=+23.702564138" watchObservedRunningTime="2026-08-04 09:08:12.445022599 +0000 UTC m=+23.702979651"
Aug 04 09:08:12.493558 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.493511    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bhl94" podStartSLOduration=3.399083907 podStartE2EDuration="23.493495677s" totalImagesPullingTime="20.09441177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:12.493289419 +0000 UTC m=+23.751246475" watchObservedRunningTime="2026-08-04 09:08:12.493495677 +0000 UTC m=+23.751452733"
Aug 04 09:08:12.507937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.507904    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dsh54" podStartSLOduration=3.428564676 podStartE2EDuration="23.507891423s" totalImagesPullingTime="20.079326747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:12.5078834 +0000 UTC m=+23.765840456" watchObservedRunningTime="2026-08-04 09:08:12.507891423 +0000 UTC m=+23.765848477"
Aug 04 09:08:12.634326 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.634277    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:12.714324 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.714299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:12.839048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:12.838987    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 09:08:13.244391 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.244271    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T09:08:12.839006806Z","UUID":"b518cdf1-ae57-48b9-9a35-f7eb8f162cdb","Handler":null,"Name":"","Endpoint":""}
Aug 04 09:08:13.245606 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.245590    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 09:08:13.245700 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.245625    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 09:08:13.289547 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.289524    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:13.289673 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:13.289624    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:13.434997 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.434967    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal" event={"ID":"bc8da4866fc9d714207467948c0e50e4","Type":"ContainerStarted","Data":"37565f8aef79f6f65618556ea7b61f16fb592e45bdd501ac95d60c0702edb436"}
Aug 04 09:08:13.436945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.436920    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw" event={"ID":"7d08c72f-7659-492c-8f62-e6455f3b6b84","Type":"ContainerStarted","Data":"6db4cf6f3a54ab6535dc8d0cd03f69fbccd082ab272d364b8dae5ff50e84fb5a"}
Aug 04 09:08:13.451841 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.451790    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-106.ec2.internal" podStartSLOduration=23.4517721 podStartE2EDuration="23.4517721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:13.451598541 +0000 UTC m=+24.709555618" watchObservedRunningTime="2026-08-04 09:08:13.4517721 +0000 UTC m=+24.709729156"
Aug 04 09:08:13.631640 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.631607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:13.662867 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.662842    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:08:13.663339 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.663324    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:08:13.690110 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:13.690070    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:14.289315 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.289285    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:14.289455 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:14.289393    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:14.289524 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.289451    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:14.289577 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:14.289534    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:14.441753 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.441698    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw" event={"ID":"7d08c72f-7659-492c-8f62-e6455f3b6b84","Type":"ContainerStarted","Data":"9c1a9d1bf19399452e7200dc94a34c78db36d00ae4b39e59a9f99f4745e5107c"}
Aug 04 09:08:14.445433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.445402    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"b667b3785e2ec669c5252d6f4aa65a02cffb420b21745a355605c5da5bbdae34"}
Aug 04 09:08:14.445749 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.445715    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:08:14.446218 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.446200    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-42hvz"
Aug 04 09:08:14.462388 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.462341    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ht6pw" podStartSLOduration=2.88897518 podStartE2EDuration="25.462323061s" totalImagesPullingTime="22.573347881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:14.462144832 +0000 UTC m=+25.720101881" watchObservedRunningTime="2026-08-04 09:08:14.462323061 +0000 UTC m=+25.720280115"
Aug 04 09:08:14.521111 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:14.521043    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:15.289527 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:15.289292    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:15.289701 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:15.289639    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:16.289858 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:16.289825    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:16.290270 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:16.289964    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:16.290270 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:16.290024    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:16.290270 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:16.290117    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:17.289930 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:17.289894    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:17.290383 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:17.290019    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:18.289542 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.289369    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:18.289809 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.289375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:18.289809 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:18.289605    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:18.289809 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:18.289679    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:18.456982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.456954    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" event={"ID":"5596782e-d881-4c0a-9d59-4bcacc8ba154","Type":"ContainerStarted","Data":"7cd0f45571c44cc42a96f3d9e3c25024b71eb70107f0d6d8c136cb14671bbdd4"}
Aug 04 09:08:18.457793 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.457183    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:08:18.459049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.459025    2558 generic.go:356] "Generic (PLEG): container finished" podID="a02136e4-8849-45d0-85db-46aac84612e9" containerID="53806a761bdf5b466c0bb360b4c2b6e3b3e4362467c50170ba0ae14fb8eb9400" exitCode=0
Aug 04 09:08:18.459160 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.459060    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerDied","Data":"53806a761bdf5b466c0bb360b4c2b6e3b3e4362467c50170ba0ae14fb8eb9400"}
Aug 04 09:08:18.473288 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.473270    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:08:18.482097 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:18.482061    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt" podStartSLOduration=9.175808114 podStartE2EDuration="29.482049387s" totalImagesPullingTime="20.306241273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:18.482019927 +0000 UTC m=+29.739976992" watchObservedRunningTime="2026-08-04 09:08:18.482049387 +0000 UTC m=+29.740006436"
Aug 04 09:08:19.290556 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.290524    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:19.290763 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:19.290658    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:19.462345 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.462315    2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 04 09:08:19.462765 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.462728    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:08:19.479305 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.479282    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:08:19.730536 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.730508    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kskw5"]
Aug 04 09:08:19.730648 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.730634    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:19.730777 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:19.730750    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:19.734367 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.734342    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9zs2"]
Aug 04 09:08:19.734482 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.734452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:19.734569 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:19.734546    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:19.735037 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.735013    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c852f"]
Aug 04 09:08:19.735157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:19.735106    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:19.735222 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:19.735200    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:20.045386 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:20.045307    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:08:20.466920 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:20.466892    2558 generic.go:356] "Generic (PLEG): container finished" podID="a02136e4-8849-45d0-85db-46aac84612e9" containerID="c79d8a4c944479d8bdbe35b629f27b0a4194ba62af9e19c889066a695d54ef6a" exitCode=0
Aug 04 09:08:20.467358 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:20.466979    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerDied","Data":"c79d8a4c944479d8bdbe35b629f27b0a4194ba62af9e19c889066a695d54ef6a"}
Aug 04 09:08:21.289416 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:21.289384    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:21.289597 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:21.289389    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:21.289597 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:21.289483    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:21.289597 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:21.289552    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:21.289785 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:21.289603    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:21.289785 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:21.289713    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:21.910336 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:21.910308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:21.910779 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:21.910423    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 09:08:21.910779 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:21.910437    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 09:08:21.910779 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:21.910446    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vckwh for pod openshift-network-diagnostics/network-check-target-l9zs2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:08:21.910779 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:21.910492    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh podName:95d2561a-6816-4955-b1e7-e8b10e974687 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:53.910479543 +0000 UTC m=+65.168436575 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vckwh" (UniqueName: "kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh") pod "network-check-target-l9zs2" (UID: "95d2561a-6816-4955-b1e7-e8b10e974687") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 09:08:22.010915 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:22.010889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:22.010915 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:22.010920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:22.011114 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:22.011014    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:08:22.011114 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:22.011049    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 09:08:22.011114 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:22.011061    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs podName:65b7539a-5c74-4647-9e2e-87e2a3961cd3 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:54.011047079 +0000 UTC m=+65.269004111 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs") pod "network-metrics-daemon-c852f" (UID: "65b7539a-5c74-4647-9e2e-87e2a3961cd3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 09:08:22.011231 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:22.011134    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret podName:e4df6d71-b2d6-4b8d-af54-36717c5c4da5 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:54.011116031 +0000 UTC m=+65.269073070 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret") pod "global-pull-secret-syncer-kskw5" (UID: "e4df6d71-b2d6-4b8d-af54-36717c5c4da5") : object "kube-system"/"original-pull-secret" not registered
Aug 04 09:08:22.472794 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:22.472762    2558 generic.go:356] "Generic (PLEG): container finished" podID="a02136e4-8849-45d0-85db-46aac84612e9" containerID="0d2807503e89ef40de6a391837d92ee591e64d6f8e91894b43acdab8777ff552" exitCode=0
Aug 04 09:08:22.472945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:22.472823    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerDied","Data":"0d2807503e89ef40de6a391837d92ee591e64d6f8e91894b43acdab8777ff552"}
Aug 04 09:08:23.153517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:23.153446    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:23.289559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:23.289523    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:23.289751 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:23.289523    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:23.289751 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:23.289651    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9zs2" podUID="95d2561a-6816-4955-b1e7-e8b10e974687"
Aug 04 09:08:23.289751 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:23.289726    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kskw5" podUID="e4df6d71-b2d6-4b8d-af54-36717c5c4da5"
Aug 04 09:08:23.289934 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:23.289523    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:23.289934 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:23.289858    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c852f" podUID="65b7539a-5c74-4647-9e2e-87e2a3961cd3"
Aug 04 09:08:23.331568 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:23.331540    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:24.304751 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.304709    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dsh54_bcac14e8-3c2b-4665-a56e-7fc16b48043f/dns-node-resolver/0.log"
Aug 04 09:08:24.474346 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.474310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:24.521648 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.521581    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-106.ec2.internal" event="NodeReady"
Aug 04 09:08:24.521795 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.521729    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 09:08:24.562872 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.562822    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76864db7fc-979sp"]
Aug 04 09:08:24.568435 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.568412    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.570859 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.570823    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 09:08:24.571137 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.571083    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 09:08:24.571609 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.571591    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-27s78\""
Aug 04 09:08:24.571701 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.571675    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5ds5r"]
Aug 04 09:08:24.572548 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.572529    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 09:08:24.575696 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.575675    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.576830 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.576809    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 09:08:24.578127 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.577701    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 09:08:24.579136 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.579025    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 09:08:24.579219 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.579135    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xnmk\""
Aug 04 09:08:24.580557 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.580525    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76864db7fc-979sp"]
Aug 04 09:08:24.581098 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.581074    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k5hsh"]
Aug 04 09:08:24.584406 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.584387    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.586780 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.586729    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 09:08:24.586909 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.586888    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 09:08:24.586980 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.586911    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-56lhf\""
Aug 04 09:08:24.587174 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.586911    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 09:08:24.589126 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.588894    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5ds5r"]
Aug 04 09:08:24.589126 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.589075    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 09:08:24.599567 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.599511    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t9sl8"]
Aug 04 09:08:24.603217 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.603200    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k5hsh"]
Aug 04 09:08:24.603327 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.603295    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:24.605623 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.605604    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 09:08:24.605693 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.605663    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 09:08:24.605693 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.605681    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 09:08:24.605982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.605962    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lkd2q\""
Aug 04 09:08:24.611892 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.611872    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9sl8"]
Aug 04 09:08:24.633014 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.632985    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:24.708774 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.708721    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:24.730722 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730698    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47752d1b-8f63-439d-b354-532f3b7ed228-cert\") pod \"ingress-canary-t9sl8\" (UID: \"47752d1b-8f63-439d-b354-532f3b7ed228\") " pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:24.730842 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-image-registry-private-configuration\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.730842 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730793    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.730842 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730818    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f48vv\" (UniqueName: \"kubernetes.io/projected/47752d1b-8f63-439d-b354-532f3b7ed228-kube-api-access-f48vv\") pod \"ingress-canary-t9sl8\" (UID: \"47752d1b-8f63-439d-b354-532f3b7ed228\") " pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:24.730976 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.730976 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730927    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/646ef199-644e-4b1c-83c6-8367c369dd61-data-volume\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.730976 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730964    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/646ef199-644e-4b1c-83c6-8367c369dd61-crio-socket\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.731089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.730985    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.731089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731035    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.731163 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d038738a-68af-43a9-8cba-e2103f662ba8-ca-trust-extracted\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.731163 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvrpj\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-kube-api-access-gvrpj\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.731163 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731146    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-tmp-dir\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.731359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwg2b\" (UniqueName: \"kubernetes.io/projected/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-kube-api-access-mwg2b\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.731359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh75h\" (UniqueName: \"kubernetes.io/projected/646ef199-644e-4b1c-83c6-8367c369dd61-kube-api-access-qh75h\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.731359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-config-volume\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.731359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731293    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-installation-pull-secrets\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.731359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-registry-certificates\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.731551 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-trusted-ca\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.731551 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.731382    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-bound-sa-token\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.832645 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-config-volume\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.832813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-installation-pull-secrets\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.832813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-registry-certificates\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.832813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-trusted-ca\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.832813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-bound-sa-token\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.832813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832763    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47752d1b-8f63-439d-b354-532f3b7ed228-cert\") pod \"ingress-canary-t9sl8\" (UID: \"47752d1b-8f63-439d-b354-532f3b7ed228\") " pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:24.832813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832783    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-image-registry-private-configuration\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.832813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832821    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f48vv\" (UniqueName: \"kubernetes.io/projected/47752d1b-8f63-439d-b354-532f3b7ed228-kube-api-access-f48vv\") pod \"ingress-canary-t9sl8\" (UID: \"47752d1b-8f63-439d-b354-532f3b7ed228\") " pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832876    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/646ef199-644e-4b1c-83c6-8367c369dd61-data-volume\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832915    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/646ef199-644e-4b1c-83c6-8367c369dd61-crio-socket\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832932    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d038738a-68af-43a9-8cba-e2103f662ba8-ca-trust-extracted\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.832981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvrpj\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-kube-api-access-gvrpj\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.833014    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-tmp-dir\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.833033    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwg2b\" (UniqueName: \"kubernetes.io/projected/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-kube-api-access-mwg2b\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.833053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.833049    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh75h\" (UniqueName: \"kubernetes.io/projected/646ef199-644e-4b1c-83c6-8367c369dd61-kube-api-access-qh75h\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.833371 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.833214    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-config-volume\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.833371 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.833334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-registry-certificates\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.833457 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.833440    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:24.833528 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.833517    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:25.333498 +0000 UTC m=+36.591455047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:24.833604 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.833589    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-tmp-dir\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.834103 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.834083    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 09:08:24.834216 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.834152    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls podName:c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f nodeName:}" failed. No retries permitted until 2026-08-04 09:08:25.334130931 +0000 UTC m=+36.592087982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls") pod "dns-default-5ds5r" (UID: "c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f") : secret "dns-default-metrics-tls" not found
Aug 04 09:08:24.834365 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.834342    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 09:08:24.834457 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.834370    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76864db7fc-979sp: secret "image-registry-tls" not found
Aug 04 09:08:24.834457 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.834430    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls podName:d038738a-68af-43a9-8cba-e2103f662ba8 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:25.334409223 +0000 UTC m=+36.592366266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls") pod "image-registry-76864db7fc-979sp" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8") : secret "image-registry-tls" not found
Aug 04 09:08:24.834580 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.834565    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 09:08:24.834631 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:24.834619    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:25.334605858 +0000 UTC m=+36.592562890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : configmap "kube-rbac-proxy" not found
Aug 04 09:08:24.834699 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.834658    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/646ef199-644e-4b1c-83c6-8367c369dd61-data-volume\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.834814 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.834776    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d038738a-68af-43a9-8cba-e2103f662ba8-ca-trust-extracted\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.834869 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.834816    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/646ef199-644e-4b1c-83c6-8367c369dd61-crio-socket\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.835324 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.835306    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-trusted-ca\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.838435 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.838412    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-image-registry-private-configuration\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.838539 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.838422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47752d1b-8f63-439d-b354-532f3b7ed228-cert\") pod \"ingress-canary-t9sl8\" (UID: \"47752d1b-8f63-439d-b354-532f3b7ed228\") " pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:24.838714 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.838694    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-installation-pull-secrets\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.846429 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.844357    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvrpj\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-kube-api-access-gvrpj\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.846429 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.844799    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh75h\" (UniqueName: \"kubernetes.io/projected/646ef199-644e-4b1c-83c6-8367c369dd61-kube-api-access-qh75h\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:24.847494 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.847437    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwg2b\" (UniqueName: \"kubernetes.io/projected/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-kube-api-access-mwg2b\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:24.847644 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.847625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-bound-sa-token\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:24.848899 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.848875    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f48vv\" (UniqueName: \"kubernetes.io/projected/47752d1b-8f63-439d-b354-532f3b7ed228-kube-api-access-f48vv\") pod \"ingress-canary-t9sl8\" (UID: \"47752d1b-8f63-439d-b354-532f3b7ed228\") " pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:24.914129 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:24.914103    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9sl8"
Aug 04 09:08:25.067271 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.067073    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9sl8"]
Aug 04 09:08:25.072860 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:25.072820    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47752d1b_8f63_439d_b354_532f3b7ed228.slice/crio-2cf277fc6e14674b29f030176fac4d604e76a71ea7e9f089b027935cf3940e66 WatchSource:0}: Error finding container 2cf277fc6e14674b29f030176fac4d604e76a71ea7e9f089b027935cf3940e66: Status 404 returned error can't find the container with id 2cf277fc6e14674b29f030176fac4d604e76a71ea7e9f089b027935cf3940e66
Aug 04 09:08:25.290185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.290071    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:25.290185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.290086    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:25.290392 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.290302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:25.293013 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.292992    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jr9km\""
Aug 04 09:08:25.293151 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.293027    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 09:08:25.293484 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.293301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djwk2\""
Aug 04 09:08:25.293484 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.293317    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 09:08:25.293484 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.293306    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 09:08:25.293685 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.293648    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 09:08:25.337646 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.337619    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.337682    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.337716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337726    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.337753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337814    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls podName:c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f nodeName:}" failed. No retries permitted until 2026-08-04 09:08:26.33779441 +0000 UTC m=+37.595751442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls") pod "dns-default-5ds5r" (UID: "c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f") : secret "dns-default-metrics-tls" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337834    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337841    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337849    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76864db7fc-979sp: secret "image-registry-tls" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337852    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337895    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:26.33788004 +0000 UTC m=+37.595837072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337911    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:26.337902948 +0000 UTC m=+37.595859979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : configmap "kube-rbac-proxy" not found
Aug 04 09:08:25.338158 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:25.337921    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls podName:d038738a-68af-43a9-8cba-e2103f662ba8 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:26.337915014 +0000 UTC m=+37.595872046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls") pod "image-registry-76864db7fc-979sp" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8") : secret "image-registry-tls" not found
Aug 04 09:08:25.479451 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.479420    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9sl8" event={"ID":"47752d1b-8f63-439d-b354-532f3b7ed228","Type":"ContainerStarted","Data":"2cf277fc6e14674b29f030176fac4d604e76a71ea7e9f089b027935cf3940e66"}
Aug 04 09:08:25.505534 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.505497    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pd75f_4ccbe261-c0cc-4deb-8575-27d331f415d3/node-ca/0.log"
Aug 04 09:08:25.528768 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.528721    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:25.633047 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:25.633020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log" limit=1024
Aug 04 09:08:26.345611 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:26.345573    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:26.345642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:26.345690    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:26.345718    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.345763    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.345836    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls podName:c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f nodeName:}" failed. No retries permitted until 2026-08-04 09:08:28.345818091 +0000 UTC m=+39.603775123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls") pod "dns-default-5ds5r" (UID: "c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f") : secret "dns-default-metrics-tls" not found
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.345887    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.345938    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:28.345921953 +0000 UTC m=+39.603879007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.345989    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.346005    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.346021    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76864db7fc-979sp: secret "image-registry-tls" not found
Aug 04 09:08:26.346058 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.346022    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:28.346010534 +0000 UTC m=+39.603967569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : configmap "kube-rbac-proxy" not found
Aug 04 09:08:26.346484 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:26.346080    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls podName:d038738a-68af-43a9-8cba-e2103f662ba8 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:28.346064279 +0000 UTC m=+39.604021317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls") pod "image-registry-76864db7fc-979sp" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8") : secret "image-registry-tls" not found
Aug 04 09:08:27.388225 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.388182    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-jrl9f"]
Aug 04 09:08:27.391298 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.391273    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.393617 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.393594    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Aug 04 09:08:27.393729 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.393618    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Aug 04 09:08:27.394526 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.394505    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Aug 04 09:08:27.394624 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.394564    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lg7fk\""
Aug 04 09:08:27.394689 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.394510    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Aug 04 09:08:27.401514 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.401462    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-jrl9f"]
Aug 04 09:08:27.487955 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.487848    2558 generic.go:356] "Generic (PLEG): container finished" podID="a02136e4-8849-45d0-85db-46aac84612e9" containerID="6996daf5bd5b48e514ab1483883b161ebeee67edcf638760b02a7a5cfa70d874" exitCode=0
Aug 04 09:08:27.487955 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.487906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerDied","Data":"6996daf5bd5b48e514ab1483883b161ebeee67edcf638760b02a7a5cfa70d874"}
Aug 04 09:08:27.556943 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.556909    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/38ec71c6-4359-4ecc-a161-1d4dce8ad544-signing-cabundle\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.557066 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.556951    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrr9q\" (UniqueName: \"kubernetes.io/projected/38ec71c6-4359-4ecc-a161-1d4dce8ad544-kube-api-access-qrr9q\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.557160 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.557142    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/38ec71c6-4359-4ecc-a161-1d4dce8ad544-signing-key\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.658450 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.657818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/38ec71c6-4359-4ecc-a161-1d4dce8ad544-signing-key\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.658450 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.658005    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/38ec71c6-4359-4ecc-a161-1d4dce8ad544-signing-cabundle\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.658450 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.658075    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrr9q\" (UniqueName: \"kubernetes.io/projected/38ec71c6-4359-4ecc-a161-1d4dce8ad544-kube-api-access-qrr9q\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.658913 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.658891    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/38ec71c6-4359-4ecc-a161-1d4dce8ad544-signing-cabundle\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.662384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.662283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/38ec71c6-4359-4ecc-a161-1d4dce8ad544-signing-key\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.667050 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.666986    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrr9q\" (UniqueName: \"kubernetes.io/projected/38ec71c6-4359-4ecc-a161-1d4dce8ad544-kube-api-access-qrr9q\") pod \"service-ca-98c88fbb5-jrl9f\" (UID: \"38ec71c6-4359-4ecc-a161-1d4dce8ad544\") " pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.701725 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.701692    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f"
Aug 04 09:08:27.855911 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:27.855879    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-jrl9f"]
Aug 04 09:08:27.860349 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:27.860321    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ec71c6_4359_4ecc_a161_1d4dce8ad544.slice/crio-761efdbd56e0b4f8de7568e676ca66975d844c74b6917caa305680d6b77c9f5b WatchSource:0}: Error finding container 761efdbd56e0b4f8de7568e676ca66975d844c74b6917caa305680d6b77c9f5b: Status 404 returned error can't find the container with id 761efdbd56e0b4f8de7568e676ca66975d844c74b6917caa305680d6b77c9f5b
Aug 04 09:08:28.365885 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:28.365840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:28.366075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:28.365908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:28.366075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:28.365945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:28.366075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:28.365971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:28.366075 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366005    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 09:08:28.366075 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366075    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls podName:c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f nodeName:}" failed. No retries permitted until 2026-08-04 09:08:32.36605337 +0000 UTC m=+43.624010407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls") pod "dns-default-5ds5r" (UID: "c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f") : secret "dns-default-metrics-tls" not found
Aug 04 09:08:28.366326 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366092    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:28.366326 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366135    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:32.366124 +0000 UTC m=+43.624081032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:28.366326 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366195    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 09:08:28.366326 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366206    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76864db7fc-979sp: secret "image-registry-tls" not found
Aug 04 09:08:28.366326 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366241    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls podName:d038738a-68af-43a9-8cba-e2103f662ba8 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:32.366229236 +0000 UTC m=+43.624186283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls") pod "image-registry-76864db7fc-979sp" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8") : secret "image-registry-tls" not found
Aug 04 09:08:28.366326 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366283    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 09:08:28.366326 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:28.366321    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:32.366310472 +0000 UTC m=+43.624267504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : configmap "kube-rbac-proxy" not found
Aug 04 09:08:28.494508 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:28.494475    2558 generic.go:356] "Generic (PLEG): container finished" podID="a02136e4-8849-45d0-85db-46aac84612e9" containerID="18febfda147daf747af530aa74adf0d8f0037fbc7ae04376d8b9bd4246679855" exitCode=0
Aug 04 09:08:28.494966 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:28.494556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerDied","Data":"18febfda147daf747af530aa74adf0d8f0037fbc7ae04376d8b9bd4246679855"}
Aug 04 09:08:28.495925 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:28.495794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f" event={"ID":"38ec71c6-4359-4ecc-a161-1d4dce8ad544","Type":"ContainerStarted","Data":"761efdbd56e0b4f8de7568e676ca66975d844c74b6917caa305680d6b77c9f5b"}
Aug 04 09:08:29.502166 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:29.502118    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" event={"ID":"a02136e4-8849-45d0-85db-46aac84612e9","Type":"ContainerStarted","Data":"ed6fc16823bedccbdc297813de56dfbdaa0821bc9c09d8945220d567658d76d3"}
Aug 04 09:08:29.504049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:29.504018    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9sl8" event={"ID":"47752d1b-8f63-439d-b354-532f3b7ed228","Type":"ContainerStarted","Data":"5acc9c57116dca141c92359a9484859333eb307970448a476e98152b3582a70d"}
Aug 04 09:08:29.547115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:29.547053    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kpmrg" podStartSLOduration=8.360611058 podStartE2EDuration="40.547033986s" totalImagesPullingTime="32.186422928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:29.546100379 +0000 UTC m=+40.804057461" watchObservedRunningTime="2026-08-04 09:08:29.547033986 +0000 UTC m=+40.804991040"
Aug 04 09:08:31.510130 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:31.510051    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f" event={"ID":"38ec71c6-4359-4ecc-a161-1d4dce8ad544","Type":"ContainerStarted","Data":"f84253a4a288e8bdc2d64e3bed0c945fc4701d501ccd115c41da6b780c50cbcb"}
Aug 04 09:08:31.527596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:31.527540    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t9sl8" podStartSLOduration=3.258567426 podStartE2EDuration="7.527523518s" totalImagesPullingTime="4.268956092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:29.575599742 +0000 UTC m=+40.833556825" watchObservedRunningTime="2026-08-04 09:08:31.527523518 +0000 UTC m=+42.785480573"
Aug 04 09:08:31.527766 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:31.527720    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-jrl9f" podStartSLOduration=1.18245775 podStartE2EDuration="4.52771315s" totalImagesPullingTime="3.3452554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:31.527218963 +0000 UTC m=+42.785176019" watchObservedRunningTime="2026-08-04 09:08:31.52771315 +0000 UTC m=+42.785670203"
Aug 04 09:08:32.397683 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:32.397635    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:32.397858 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:32.397703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:32.397858 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:32.397748    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:32.397858 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397798    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 09:08:32.397858 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397812    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 09:08:32.397858 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397819    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76864db7fc-979sp: secret "image-registry-tls" not found
Aug 04 09:08:32.397858 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:32.397833    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:32.398048 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397875    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:40.397854225 +0000 UTC m=+51.655811291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : configmap "kube-rbac-proxy" not found
Aug 04 09:08:32.398048 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397902    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls podName:d038738a-68af-43a9-8cba-e2103f662ba8 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:40.397895356 +0000 UTC m=+51.655852388 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls") pod "image-registry-76864db7fc-979sp" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8") : secret "image-registry-tls" not found
Aug 04 09:08:32.398048 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397912    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:32.398048 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397924    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 09:08:32.398048 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397969    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:40.397951396 +0000 UTC m=+51.655908433 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:32.398048 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:32.397988    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls podName:c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f nodeName:}" failed. No retries permitted until 2026-08-04 09:08:40.397979119 +0000 UTC m=+51.655936156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls") pod "dns-default-5ds5r" (UID: "c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f") : secret "dns-default-metrics-tls" not found
Aug 04 09:08:40.464455 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.464420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:40.465033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.464475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:40.465033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.464516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:40.465033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.464543    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:40.465033 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:40.464658    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:40.465033 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:08:40.464724    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls podName:646ef199-644e-4b1c-83c6-8367c369dd61 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:56.464704898 +0000 UTC m=+67.722661931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5hsh" (UID: "646ef199-644e-4b1c-83c6-8367c369dd61") : secret "insights-runtime-extractor-tls" not found
Aug 04 09:08:40.465033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.465029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/646ef199-644e-4b1c-83c6-8367c369dd61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:40.466922 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.466894    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f-metrics-tls\") pod \"dns-default-5ds5r\" (UID: \"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f\") " pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:40.467047 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.467029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"image-registry-76864db7fc-979sp\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") " pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:40.484070 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.484042    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:40.492439 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.492414    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:40.627705 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.627671    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76864db7fc-979sp"]
Aug 04 09:08:40.641087 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:40.641065    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5ds5r"]
Aug 04 09:08:40.643912 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:40.643881    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5bae8b5_8e93_46c8_9af4_f180ce3b8c5f.slice/crio-586362ea2e05c508a117997e92007af9cc68c8579b74062c640c5d2d09c6c8fe WatchSource:0}: Error finding container 586362ea2e05c508a117997e92007af9cc68c8579b74062c640c5d2d09c6c8fe: Status 404 returned error can't find the container with id 586362ea2e05c508a117997e92007af9cc68c8579b74062c640c5d2d09c6c8fe
Aug 04 09:08:41.534970 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:41.534928    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ds5r" event={"ID":"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f","Type":"ContainerStarted","Data":"586362ea2e05c508a117997e92007af9cc68c8579b74062c640c5d2d09c6c8fe"}
Aug 04 09:08:41.537411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:41.537382    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76864db7fc-979sp" event={"ID":"d038738a-68af-43a9-8cba-e2103f662ba8","Type":"ContainerStarted","Data":"240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed"}
Aug 04 09:08:41.537531 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:41.537432    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76864db7fc-979sp" event={"ID":"d038738a-68af-43a9-8cba-e2103f662ba8","Type":"ContainerStarted","Data":"0d1d1cd5244d2963c08f2042e1d1319a0d36b9a841d56014be731dd0e1ed64bb"}
Aug 04 09:08:41.537531 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:41.537513    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:08:41.558249 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:41.558202    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76864db7fc-979sp" podStartSLOduration=38.558181938 podStartE2EDuration="38.558181938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:41.556898572 +0000 UTC m=+52.814855625" watchObservedRunningTime="2026-08-04 09:08:41.558181938 +0000 UTC m=+52.816138993"
Aug 04 09:08:43.544465 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:43.544383    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ds5r" event={"ID":"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f","Type":"ContainerStarted","Data":"a8c0a39a8815a1f3def16b2bc455676cd59144ceb1094440fff980cdb118734b"}
Aug 04 09:08:43.544465 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:43.544442    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ds5r" event={"ID":"c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f","Type":"ContainerStarted","Data":"36e83cd6bf6d703dea97bcf9dbb762bed2e557f75e6fc3572cdb63fbf9ea5514"}
Aug 04 09:08:43.544872 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:43.544472    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:43.561401 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:43.561352    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5ds5r" podStartSLOduration=17.358827152 podStartE2EDuration="19.561336923s" totalImagesPullingTime="2.202509771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:43.560592747 +0000 UTC m=+54.818549801" watchObservedRunningTime="2026-08-04 09:08:43.561336923 +0000 UTC m=+54.819293977"
Aug 04 09:08:47.227580 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.227543    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-f5k8d"]
Aug 04 09:08:47.232105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.232088    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-f5k8d"
Aug 04 09:08:47.235826 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.235785    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 04 09:08:47.240195 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.240165    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 04 09:08:47.242883 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.242866    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-q77wd\""
Aug 04 09:08:47.281246 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.281224    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-f5k8d"]
Aug 04 09:08:47.314910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.314884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmhp6\" (UniqueName: \"kubernetes.io/projected/7fc31d53-4275-4f0a-a52a-a6ddf8161869-kube-api-access-wmhp6\") pod \"downloads-77c974ffcf-f5k8d\" (UID: \"7fc31d53-4275-4f0a-a52a-a6ddf8161869\") " pod="openshift-console/downloads-77c974ffcf-f5k8d"
Aug 04 09:08:47.337943 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.337912    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76864db7fc-979sp"]
Aug 04 09:08:47.415717 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.415684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmhp6\" (UniqueName: \"kubernetes.io/projected/7fc31d53-4275-4f0a-a52a-a6ddf8161869-kube-api-access-wmhp6\") pod \"downloads-77c974ffcf-f5k8d\" (UID: \"7fc31d53-4275-4f0a-a52a-a6ddf8161869\") " pod="openshift-console/downloads-77c974ffcf-f5k8d"
Aug 04 09:08:47.481324 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.481269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmhp6\" (UniqueName: \"kubernetes.io/projected/7fc31d53-4275-4f0a-a52a-a6ddf8161869-kube-api-access-wmhp6\") pod \"downloads-77c974ffcf-f5k8d\" (UID: \"7fc31d53-4275-4f0a-a52a-a6ddf8161869\") " pod="openshift-console/downloads-77c974ffcf-f5k8d"
Aug 04 09:08:47.521322 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.521302    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8497dfc489-xnv7h"]
Aug 04 09:08:47.524180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.524165    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.544218 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.544195    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-f5k8d"
Aug 04 09:08:47.558352 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.558333    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8497dfc489-xnv7h"]
Aug 04 09:08:47.617403 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617369    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fee394f-ab08-4450-8ed2-bae1f697907c-registry-certificates\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.617529 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617422    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fee394f-ab08-4450-8ed2-bae1f697907c-ca-trust-extracted\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.617529 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617499    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-registry-tls\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.617636 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgp2g\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-kube-api-access-zgp2g\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.617636 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-bound-sa-token\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.617636 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617584    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fee394f-ab08-4450-8ed2-bae1f697907c-installation-pull-secrets\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.617752 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fee394f-ab08-4450-8ed2-bae1f697907c-image-registry-private-configuration\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.617752 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.617707    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fee394f-ab08-4450-8ed2-bae1f697907c-trusted-ca\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.688070 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.688039    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-f5k8d"]
Aug 04 09:08:47.691103 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:47.691078    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc31d53_4275_4f0a_a52a_a6ddf8161869.slice/crio-a6f4132921c67123b589f67d4912631a21a4ca08bded461d696504f4c7143b30 WatchSource:0}: Error finding container a6f4132921c67123b589f67d4912631a21a4ca08bded461d696504f4c7143b30: Status 404 returned error can't find the container with id a6f4132921c67123b589f67d4912631a21a4ca08bded461d696504f4c7143b30
Aug 04 09:08:47.718992 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.718969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fee394f-ab08-4450-8ed2-bae1f697907c-ca-trust-extracted\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-registry-tls\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgp2g\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-kube-api-access-zgp2g\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-bound-sa-token\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fee394f-ab08-4450-8ed2-bae1f697907c-installation-pull-secrets\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fee394f-ab08-4450-8ed2-bae1f697907c-image-registry-private-configuration\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fee394f-ab08-4450-8ed2-bae1f697907c-trusted-ca\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fee394f-ab08-4450-8ed2-bae1f697907c-registry-certificates\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.719419 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.719386    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fee394f-ab08-4450-8ed2-bae1f697907c-ca-trust-extracted\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.720034 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.720010    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fee394f-ab08-4450-8ed2-bae1f697907c-registry-certificates\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.720451 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.720426    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fee394f-ab08-4450-8ed2-bae1f697907c-trusted-ca\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.721572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.721546    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fee394f-ab08-4450-8ed2-bae1f697907c-installation-pull-secrets\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.721663 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.721643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-registry-tls\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.721704 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.721645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fee394f-ab08-4450-8ed2-bae1f697907c-image-registry-private-configuration\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.733233 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.733188    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgp2g\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-kube-api-access-zgp2g\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.733356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.733338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fee394f-ab08-4450-8ed2-bae1f697907c-bound-sa-token\") pod \"image-registry-8497dfc489-xnv7h\" (UID: \"1fee394f-ab08-4450-8ed2-bae1f697907c\") " pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.832463 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.832432    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:47.987483 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:47.987407    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8497dfc489-xnv7h"]
Aug 04 09:08:47.994258 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:47.994228    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fee394f_ab08_4450_8ed2_bae1f697907c.slice/crio-257a7fa2bc419516f8acac022466f0338beebf23c82ebed04b27301da21002d2 WatchSource:0}: Error finding container 257a7fa2bc419516f8acac022466f0338beebf23c82ebed04b27301da21002d2: Status 404 returned error can't find the container with id 257a7fa2bc419516f8acac022466f0338beebf23c82ebed04b27301da21002d2
Aug 04 09:08:48.560581 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:48.560531    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-f5k8d" event={"ID":"7fc31d53-4275-4f0a-a52a-a6ddf8161869","Type":"ContainerStarted","Data":"a6f4132921c67123b589f67d4912631a21a4ca08bded461d696504f4c7143b30"}
Aug 04 09:08:48.562343 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:48.562315    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8497dfc489-xnv7h" event={"ID":"1fee394f-ab08-4450-8ed2-bae1f697907c","Type":"ContainerStarted","Data":"f4566014c55ea3a8b79e4c6615a0a20a268cf671289051a98077b272c277c727"}
Aug 04 09:08:48.562453 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:48.562365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8497dfc489-xnv7h" event={"ID":"1fee394f-ab08-4450-8ed2-bae1f697907c","Type":"ContainerStarted","Data":"257a7fa2bc419516f8acac022466f0338beebf23c82ebed04b27301da21002d2"}
Aug 04 09:08:48.562505 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:48.562448    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:08:48.588433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:48.588375    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8497dfc489-xnv7h" podStartSLOduration=1.5883577 podStartE2EDuration="1.5883577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:48.586904517 +0000 UTC m=+59.844861573" watchObservedRunningTime="2026-08-04 09:08:48.5883577 +0000 UTC m=+59.846314757"
Aug 04 09:08:51.481065 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:51.481033    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsvrt"
Aug 04 09:08:53.549278 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:53.549248    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5ds5r"
Aug 04 09:08:53.973501 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:53.973467    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:53.976007 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:53.975980    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 09:08:53.986336 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:53.986314    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 09:08:53.997841 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:53.997805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vckwh\" (UniqueName: \"kubernetes.io/projected/95d2561a-6816-4955-b1e7-e8b10e974687-kube-api-access-vckwh\") pod \"network-check-target-l9zs2\" (UID: \"95d2561a-6816-4955-b1e7-e8b10e974687\") " pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:54.074045 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.074007    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:54.074205 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.074066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:54.076817 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.076792    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 09:08:54.076945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.076798    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 09:08:54.087052 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.087030    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7539a-5c74-4647-9e2e-87e2a3961cd3-metrics-certs\") pod \"network-metrics-daemon-c852f\" (UID: \"65b7539a-5c74-4647-9e2e-87e2a3961cd3\") " pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:54.087165 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.087035    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4df6d71-b2d6-4b8d-af54-36717c5c4da5-original-pull-secret\") pod \"global-pull-secret-syncer-kskw5\" (UID: \"e4df6d71-b2d6-4b8d-af54-36717c5c4da5\") " pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:54.102339 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.102309    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kskw5"
Aug 04 09:08:54.112168 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.112147    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djwk2\""
Aug 04 09:08:54.117866 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.117663    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jr9km\""
Aug 04 09:08:54.119793 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.119767    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c852f"
Aug 04 09:08:54.126272 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.126242    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:08:54.280163 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.280129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kskw5"]
Aug 04 09:08:54.284349 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:54.284326    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4df6d71_b2d6_4b8d_af54_36717c5c4da5.slice/crio-c414a05eefcddf1c26716d22ce75cf33e9d58b3075ab3d00e5a71110b0dd9191 WatchSource:0}: Error finding container c414a05eefcddf1c26716d22ce75cf33e9d58b3075ab3d00e5a71110b0dd9191: Status 404 returned error can't find the container with id c414a05eefcddf1c26716d22ce75cf33e9d58b3075ab3d00e5a71110b0dd9191
Aug 04 09:08:54.504368 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.504279    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9zs2"]
Aug 04 09:08:54.506545 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.506523    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c852f"]
Aug 04 09:08:54.508288 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:54.508258    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d2561a_6816_4955_b1e7_e8b10e974687.slice/crio-200ad707ac70d34bb9bf8cfc6a61b220a62386c6971d0a158ef461b96f5d2bbf WatchSource:0}: Error finding container 200ad707ac70d34bb9bf8cfc6a61b220a62386c6971d0a158ef461b96f5d2bbf: Status 404 returned error can't find the container with id 200ad707ac70d34bb9bf8cfc6a61b220a62386c6971d0a158ef461b96f5d2bbf
Aug 04 09:08:54.509709 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:54.509688    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b7539a_5c74_4647_9e2e_87e2a3961cd3.slice/crio-4232a732133e181d8a48d520a128e68c9d1a7da79a30bc3c8f8415640eb20641 WatchSource:0}: Error finding container 4232a732133e181d8a48d520a128e68c9d1a7da79a30bc3c8f8415640eb20641: Status 404 returned error can't find the container with id 4232a732133e181d8a48d520a128e68c9d1a7da79a30bc3c8f8415640eb20641
Aug 04 09:08:54.583304 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.583263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c852f" event={"ID":"65b7539a-5c74-4647-9e2e-87e2a3961cd3","Type":"ContainerStarted","Data":"4232a732133e181d8a48d520a128e68c9d1a7da79a30bc3c8f8415640eb20641"}
Aug 04 09:08:54.584423 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.584391    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9zs2" event={"ID":"95d2561a-6816-4955-b1e7-e8b10e974687","Type":"ContainerStarted","Data":"200ad707ac70d34bb9bf8cfc6a61b220a62386c6971d0a158ef461b96f5d2bbf"}
Aug 04 09:08:54.585646 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:54.585621    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kskw5" event={"ID":"e4df6d71-b2d6-4b8d-af54-36717c5c4da5","Type":"ContainerStarted","Data":"c414a05eefcddf1c26716d22ce75cf33e9d58b3075ab3d00e5a71110b0dd9191"}
Aug 04 09:08:56.495721 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.495675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:56.506146 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.506104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/646ef199-644e-4b1c-83c6-8367c369dd61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5hsh\" (UID: \"646ef199-644e-4b1c-83c6-8367c369dd61\") " pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:56.600766 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.600326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c852f" event={"ID":"65b7539a-5c74-4647-9e2e-87e2a3961cd3","Type":"ContainerStarted","Data":"019b0c1ea27d69a158f692409d89d13cc57c067ee33c266417b299a0d3a199ff"}
Aug 04 09:08:56.600766 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.600441    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c852f" event={"ID":"65b7539a-5c74-4647-9e2e-87e2a3961cd3","Type":"ContainerStarted","Data":"24f13ae913e348900e2a57e437205859c5eee77b16e1f6fa1a97f589877b19b4"}
Aug 04 09:08:56.700368 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.700337    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-56lhf\""
Aug 04 09:08:56.708490 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.708462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k5hsh"
Aug 04 09:08:56.894251 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.894152    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c852f" podStartSLOduration=66.405513162 podStartE2EDuration="1m7.89412915s" totalImagesPullingTime="1.488615988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:56.6269494 +0000 UTC m=+67.884906456" watchObservedRunningTime="2026-08-04 09:08:56.89412915 +0000 UTC m=+68.152086205"
Aug 04 09:08:56.894533 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:56.894506    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k5hsh"]
Aug 04 09:08:56.898402 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:08:56.898374    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646ef199_644e_4b1c_83c6_8367c369dd61.slice/crio-33c1eea686664243307f29e63e96ffda25c62a3f0dc7512e38d729e8fb70a772 WatchSource:0}: Error finding container 33c1eea686664243307f29e63e96ffda25c62a3f0dc7512e38d729e8fb70a772: Status 404 returned error can't find the container with id 33c1eea686664243307f29e63e96ffda25c62a3f0dc7512e38d729e8fb70a772
Aug 04 09:08:57.344520 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:57.344488    2558 patch_prober.go:28] interesting pod/image-registry-76864db7fc-979sp container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Aug 04 09:08:57.344694 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:57.344538    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76864db7fc-979sp" podUID="d038738a-68af-43a9-8cba-e2103f662ba8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 09:08:57.605433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:57.605342    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5hsh" event={"ID":"646ef199-644e-4b1c-83c6-8367c369dd61","Type":"ContainerStarted","Data":"f0fb91054fa352b481b593508e8f777cee1df7c2426a2456d1ebb0fcfc3cb8a3"}
Aug 04 09:08:57.605433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:08:57.605420    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5hsh" event={"ID":"646ef199-644e-4b1c-83c6-8367c369dd61","Type":"ContainerStarted","Data":"33c1eea686664243307f29e63e96ffda25c62a3f0dc7512e38d729e8fb70a772"}
Aug 04 09:09:07.344717 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.344682    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:09:07.648755 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.648640    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-f5k8d" event={"ID":"7fc31d53-4275-4f0a-a52a-a6ddf8161869","Type":"ContainerStarted","Data":"0a5ab80f4ab66326983e91201985afc67031a802abdac92add1c18be2d04458d"}
Aug 04 09:09:07.648960 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.648928    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-f5k8d"
Aug 04 09:09:07.650711 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.650666    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9zs2" event={"ID":"95d2561a-6816-4955-b1e7-e8b10e974687","Type":"ContainerStarted","Data":"d9c47bab2758c6d3453f96850053d08096e3321acf492105e79330ccf5a7574a"}
Aug 04 09:09:07.650878 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.650770    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:09:07.652534 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.652510    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kskw5" event={"ID":"e4df6d71-b2d6-4b8d-af54-36717c5c4da5","Type":"ContainerStarted","Data":"32c2b3fb95701f73a587e7179fc6ec9c1a07ebf405fe262b4c36e61464b4c2c9"}
Aug 04 09:09:07.654459 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.654438    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5hsh" event={"ID":"646ef199-644e-4b1c-83c6-8367c369dd61","Type":"ContainerStarted","Data":"b1ea8550f6bb998466301f84658fb407b0eb901b4ac4659a20008de988a8622c"}
Aug 04 09:09:07.660813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.660790    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-f5k8d"
Aug 04 09:09:07.665816 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.665770    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-f5k8d" podStartSLOduration=1.5184493589999999 podStartE2EDuration="20.665756744s" totalImagesPullingTime="19.147307385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:07.664428446 +0000 UTC m=+78.922385502" watchObservedRunningTime="2026-08-04 09:09:07.665756744 +0000 UTC m=+78.923713789"
Aug 04 09:09:07.696867 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.696784    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kskw5" podStartSLOduration=66.142596968 podStartE2EDuration="1m18.696766868s" totalImagesPullingTime="12.5541699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:07.695000493 +0000 UTC m=+78.952957547" watchObservedRunningTime="2026-08-04 09:09:07.696766868 +0000 UTC m=+78.954723926"
Aug 04 09:09:07.712405 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:07.711103    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l9zs2" podStartSLOduration=66.38188327 podStartE2EDuration="1m18.711083528s" totalImagesPullingTime="12.329200258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:07.710775129 +0000 UTC m=+78.968732183" watchObservedRunningTime="2026-08-04 09:09:07.711083528 +0000 UTC m=+78.969040582"
Aug 04 09:09:09.570701 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:09.570669    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8497dfc489-xnv7h"
Aug 04 09:09:09.665675 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:09.665641    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5hsh" event={"ID":"646ef199-644e-4b1c-83c6-8367c369dd61","Type":"ContainerStarted","Data":"7110cb673e6162ee1462f843674935f13d324600b2406801a78264c7e8a0835e"}
Aug 04 09:09:09.688766 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:09.688705    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k5hsh" podStartSLOduration=33.735364741 podStartE2EDuration="45.688688663s" totalImagesPullingTime="11.953323922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:09.687382528 +0000 UTC m=+80.945339583" watchObservedRunningTime="2026-08-04 09:09:09.688688663 +0000 UTC m=+80.946645717"
Aug 04 09:09:12.356484 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.356332    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76864db7fc-979sp" podUID="d038738a-68af-43a9-8cba-e2103f662ba8" containerName="registry" containerID="cri-o://240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed" gracePeriod=30
Aug 04 09:09:12.656382 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.656351    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:09:12.678150 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.678120    2558 generic.go:356] "Generic (PLEG): container finished" podID="d038738a-68af-43a9-8cba-e2103f662ba8" containerID="240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed" exitCode=0
Aug 04 09:09:12.678296 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.678183    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76864db7fc-979sp" event={"ID":"d038738a-68af-43a9-8cba-e2103f662ba8","Type":"ContainerDied","Data":"240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed"}
Aug 04 09:09:12.678296 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.678189    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76864db7fc-979sp"
Aug 04 09:09:12.678296 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.678238    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76864db7fc-979sp" event={"ID":"d038738a-68af-43a9-8cba-e2103f662ba8","Type":"ContainerDied","Data":"0d1d1cd5244d2963c08f2042e1d1319a0d36b9a841d56014be731dd0e1ed64bb"}
Aug 04 09:09:12.678296 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.678266    2558 scope.go:117] "RemoveContainer" containerID="240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed"
Aug 04 09:09:12.687444 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.687424    2558 scope.go:117] "RemoveContainer" containerID="240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed"
Aug 04 09:09:12.687797 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:09:12.687771    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed\": container with ID starting with 240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed not found: ID does not exist" containerID="240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed"
Aug 04 09:09:12.687900 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.687806    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed"} err="failed to get container status \"240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed\": rpc error: code = NotFound desc = could not find container \"240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed\": container with ID starting with 240fedd62b4cbcd6aaa7d379904363fe9740715f3cd67dbb3ae0ee58d17a15ed not found: ID does not exist"
Aug 04 09:09:12.727897 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.727866    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d038738a-68af-43a9-8cba-e2103f662ba8-ca-trust-extracted\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728061 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.727910    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-registry-certificates\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728061 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.727952    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-bound-sa-token\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728061 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.727984    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-image-registry-private-configuration\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728061 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.728009    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728061 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.728037    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvrpj\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-kube-api-access-gvrpj\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728322 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.728068    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-installation-pull-secrets\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728322 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.728096    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-trusted-ca\") pod \"d038738a-68af-43a9-8cba-e2103f662ba8\" (UID: \"d038738a-68af-43a9-8cba-e2103f662ba8\") "
Aug 04 09:09:12.728590 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.728550    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:09:12.728952 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.728660    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:09:12.733346 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.732828    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:09:12.733346 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.733311    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-kube-api-access-gvrpj" (OuterVolumeSpecName: "kube-api-access-gvrpj") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "kube-api-access-gvrpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:09:12.733577 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.733544    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:09:12.733894 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.733859    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:09:12.736250 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.736147    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:09:12.739214 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.739185    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d038738a-68af-43a9-8cba-e2103f662ba8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d038738a-68af-43a9-8cba-e2103f662ba8" (UID: "d038738a-68af-43a9-8cba-e2103f662ba8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:09:12.828717 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828681    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d038738a-68af-43a9-8cba-e2103f662ba8-ca-trust-extracted\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:12.828717 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828713    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-registry-certificates\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:12.828987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828730    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-bound-sa-token\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:12.828987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828767    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-image-registry-private-configuration\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:12.828987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828783    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-registry-tls\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:12.828987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828795    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvrpj\" (UniqueName: \"kubernetes.io/projected/d038738a-68af-43a9-8cba-e2103f662ba8-kube-api-access-gvrpj\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:12.828987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828808    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d038738a-68af-43a9-8cba-e2103f662ba8-installation-pull-secrets\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:12.828987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:12.828822    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d038738a-68af-43a9-8cba-e2103f662ba8-trusted-ca\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:09:13.014730 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:13.014690    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76864db7fc-979sp"]
Aug 04 09:09:13.024672 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:13.024644    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76864db7fc-979sp"]
Aug 04 09:09:13.295132 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:13.295061    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d038738a-68af-43a9-8cba-e2103f662ba8" path="/var/lib/kubelet/pods/d038738a-68af-43a9-8cba-e2103f662ba8/volumes"
Aug 04 09:09:17.681591 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.681557    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"]
Aug 04 09:09:17.682072 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.681777    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d038738a-68af-43a9-8cba-e2103f662ba8" containerName="registry"
Aug 04 09:09:17.682072 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.681787    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d038738a-68af-43a9-8cba-e2103f662ba8" containerName="registry"
Aug 04 09:09:17.682072 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.681832    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d038738a-68af-43a9-8cba-e2103f662ba8" containerName="registry"
Aug 04 09:09:17.713463 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.713431    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"]
Aug 04 09:09:17.713600 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.713535    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.716300 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.716279    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 09:09:17.716944 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.716924    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Aug 04 09:09:17.716944 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.716946    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 09:09:17.717105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.716962    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 09:09:17.717105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.717057    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t9zrt\""
Aug 04 09:09:17.717105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.717065    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Aug 04 09:09:17.865965 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.865934    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/873d73b6-c0cd-4f42-8af3-cbfdf390500c-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.865965 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.865967    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cpsl\" (UniqueName: \"kubernetes.io/projected/873d73b6-c0cd-4f42-8af3-cbfdf390500c-kube-api-access-7cpsl\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.866164 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.866001    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/873d73b6-c0cd-4f42-8af3-cbfdf390500c-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.866164 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.866050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/873d73b6-c0cd-4f42-8af3-cbfdf390500c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.966658 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.966582    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/873d73b6-c0cd-4f42-8af3-cbfdf390500c-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.966658 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.966620    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/873d73b6-c0cd-4f42-8af3-cbfdf390500c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.966658 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.966651    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/873d73b6-c0cd-4f42-8af3-cbfdf390500c-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.966907 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.966777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cpsl\" (UniqueName: \"kubernetes.io/projected/873d73b6-c0cd-4f42-8af3-cbfdf390500c-kube-api-access-7cpsl\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.967543 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.967473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/873d73b6-c0cd-4f42-8af3-cbfdf390500c-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.969183 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.969160    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/873d73b6-c0cd-4f42-8af3-cbfdf390500c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.969285 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.969272    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/873d73b6-c0cd-4f42-8af3-cbfdf390500c-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:17.976245 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:17.976223    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cpsl\" (UniqueName: \"kubernetes.io/projected/873d73b6-c0cd-4f42-8af3-cbfdf390500c-kube-api-access-7cpsl\") pod \"prometheus-operator-758d7cb874-gvqkx\" (UID: \"873d73b6-c0cd-4f42-8af3-cbfdf390500c\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:18.022939 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:18.022897    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"
Aug 04 09:09:18.165713 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:18.165681    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-gvqkx"]
Aug 04 09:09:18.168754 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:09:18.168704    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod873d73b6_c0cd_4f42_8af3_cbfdf390500c.slice/crio-f6bfd1a0bedf5fc74208dd0ad20fe660b8b033b88c9b8eda49e8706020681684 WatchSource:0}: Error finding container f6bfd1a0bedf5fc74208dd0ad20fe660b8b033b88c9b8eda49e8706020681684: Status 404 returned error can't find the container with id f6bfd1a0bedf5fc74208dd0ad20fe660b8b033b88c9b8eda49e8706020681684
Aug 04 09:09:18.701126 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:18.701094    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx" event={"ID":"873d73b6-c0cd-4f42-8af3-cbfdf390500c","Type":"ContainerStarted","Data":"f6bfd1a0bedf5fc74208dd0ad20fe660b8b033b88c9b8eda49e8706020681684"}
Aug 04 09:09:20.709255 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:20.709220    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx" event={"ID":"873d73b6-c0cd-4f42-8af3-cbfdf390500c","Type":"ContainerStarted","Data":"a3370567c2a05de713c8ad176e24c2a07c3ef7248d12565963bce2f1e16f7b96"}
Aug 04 09:09:20.709620 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:20.709271    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx" event={"ID":"873d73b6-c0cd-4f42-8af3-cbfdf390500c","Type":"ContainerStarted","Data":"11a8a0b6ab5a8b0bffd694521331d14804fc9b3e288f146f85b85d1cec7e17f5"}
Aug 04 09:09:20.726341 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:20.726295    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-gvqkx" podStartSLOduration=1.860417522 podStartE2EDuration="3.726281515s" totalImagesPullingTime="1.865863993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:20.726223926 +0000 UTC m=+91.984180980" watchObservedRunningTime="2026-08-04 09:09:20.726281515 +0000 UTC m=+91.984238569"
Aug 04 09:09:27.073146 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.073106    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6jvw5"]
Aug 04 09:09:27.094959 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.094931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.097488 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.097464    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 09:09:27.097610 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.097532    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 09:09:27.097610 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.097589    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 09:09:27.097812 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.097786    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lhsjg\""
Aug 04 09:09:27.230294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-tls\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230457 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-textfile\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230457 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230333    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hpx5\" (UniqueName: \"kubernetes.io/projected/a409abeb-07c9-43f4-af5e-f9590d532649-kube-api-access-9hpx5\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230457 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-sys\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230628 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230628 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230492    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-wtmp\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230628 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a409abeb-07c9-43f4-af5e-f9590d532649-metrics-client-ca\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230628 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230564    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-root\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.230628 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.230612    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-accelerators-collector-config\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.331543 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.331519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-accelerators-collector-config\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.331705 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.331559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-tls\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.331705 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.331578    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-textfile\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.331849 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:09:27.331756    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 04 09:09:27.331849 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:09:27.331803    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-tls podName:a409abeb-07c9-43f4-af5e-f9590d532649 nodeName:}" failed. No retries permitted until 2026-08-04 09:09:27.831786024 +0000 UTC m=+99.089743056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-tls") pod "node-exporter-6jvw5" (UID: "a409abeb-07c9-43f4-af5e-f9590d532649") : secret "node-exporter-tls" not found
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.331964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hpx5\" (UniqueName: \"kubernetes.io/projected/a409abeb-07c9-43f4-af5e-f9590d532649-kube-api-access-9hpx5\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-sys\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-textfile\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332097    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-wtmp\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332116    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-sys\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a409abeb-07c9-43f4-af5e-f9590d532649-metrics-client-ca\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332188    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-accelerators-collector-config\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332194    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-root\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332218    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-root\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332269 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-wtmp\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.332929 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.332717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a409abeb-07c9-43f4-af5e-f9590d532649-metrics-client-ca\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.335018 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.334995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.341847 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.341824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hpx5\" (UniqueName: \"kubernetes.io/projected/a409abeb-07c9-43f4-af5e-f9590d532649-kube-api-access-9hpx5\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.836054 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.836027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-tls\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:27.838473 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:27.838453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a409abeb-07c9-43f4-af5e-f9590d532649-node-exporter-tls\") pod \"node-exporter-6jvw5\" (UID: \"a409abeb-07c9-43f4-af5e-f9590d532649\") " pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:28.004653 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:28.004622    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6jvw5"
Aug 04 09:09:28.036458 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:09:28.036429    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda409abeb_07c9_43f4_af5e_f9590d532649.slice/crio-e64b7a2f75c06466049552eaf751f0606cce425e2978d76b1aac8dc845076d0a WatchSource:0}: Error finding container e64b7a2f75c06466049552eaf751f0606cce425e2978d76b1aac8dc845076d0a: Status 404 returned error can't find the container with id e64b7a2f75c06466049552eaf751f0606cce425e2978d76b1aac8dc845076d0a
Aug 04 09:09:28.736729 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:28.736683    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jvw5" event={"ID":"a409abeb-07c9-43f4-af5e-f9590d532649","Type":"ContainerStarted","Data":"e64b7a2f75c06466049552eaf751f0606cce425e2978d76b1aac8dc845076d0a"}
Aug 04 09:09:30.747448 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:30.747410    2558 generic.go:356] "Generic (PLEG): container finished" podID="a409abeb-07c9-43f4-af5e-f9590d532649" containerID="9db00f4908673e1a6f281631dcf2311190b5f285fc7c6205edad55a94bc43e49" exitCode=0
Aug 04 09:09:30.747448 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:30.747450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jvw5" event={"ID":"a409abeb-07c9-43f4-af5e-f9590d532649","Type":"ContainerDied","Data":"9db00f4908673e1a6f281631dcf2311190b5f285fc7c6205edad55a94bc43e49"}
Aug 04 09:09:31.346431 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.346399    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f8f695f98-pcpbc"]
Aug 04 09:09:31.363128 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.363099    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f8f695f98-pcpbc"]
Aug 04 09:09:31.363287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.363209    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.366969 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.366944    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 04 09:09:31.367124 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.366944    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bbv5c\""
Aug 04 09:09:31.367124 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.367055    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Aug 04 09:09:31.367386 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.367365    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f2od1lsu5f5bs\""
Aug 04 09:09:31.367502 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.367384    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Aug 04 09:09:31.367571 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.367536    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Aug 04 09:09:31.462639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.462613    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/50757688-799c-4ce8-9781-f454b95b74c7-audit-log\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.462814 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.462645    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-secret-metrics-server-client-certs\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.462814 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.462698    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50757688-799c-4ce8-9781-f454b95b74c7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.462901 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.462830    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/50757688-799c-4ce8-9781-f454b95b74c7-metrics-server-audit-profiles\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.462901 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.462876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-client-ca-bundle\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.462977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.462955    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-secret-metrics-server-tls\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.463024 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.463007    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwmlv\" (UniqueName: \"kubernetes.io/projected/50757688-799c-4ce8-9781-f454b95b74c7-kube-api-access-lwmlv\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.563504 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.563474    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/50757688-799c-4ce8-9781-f454b95b74c7-metrics-server-audit-profiles\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.563625 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.563513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-client-ca-bundle\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.563625 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.563533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-secret-metrics-server-tls\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.563694 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.563652    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwmlv\" (UniqueName: \"kubernetes.io/projected/50757688-799c-4ce8-9781-f454b95b74c7-kube-api-access-lwmlv\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.563694 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.563685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/50757688-799c-4ce8-9781-f454b95b74c7-audit-log\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.563793 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.563711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-secret-metrics-server-client-certs\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.563793 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.563782    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50757688-799c-4ce8-9781-f454b95b74c7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.564062 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.564031    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/50757688-799c-4ce8-9781-f454b95b74c7-audit-log\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.564508 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.564487    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/50757688-799c-4ce8-9781-f454b95b74c7-metrics-server-audit-profiles\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.564606 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.564575    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50757688-799c-4ce8-9781-f454b95b74c7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.566213 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.566191    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-client-ca-bundle\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.566296 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.566236    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-secret-metrics-server-tls\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.566341 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.566310    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/50757688-799c-4ce8-9781-f454b95b74c7-secret-metrics-server-client-certs\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.573202 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.573183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwmlv\" (UniqueName: \"kubernetes.io/projected/50757688-799c-4ce8-9781-f454b95b74c7-kube-api-access-lwmlv\") pod \"metrics-server-5f8f695f98-pcpbc\" (UID: \"50757688-799c-4ce8-9781-f454b95b74c7\") " pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.673026 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.672967    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:31.755137 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.753830    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jvw5" event={"ID":"a409abeb-07c9-43f4-af5e-f9590d532649","Type":"ContainerStarted","Data":"1b84e9c9a498675417b89dd728a2c3af5c8e20d029a05903ae0cddd80750af13"}
Aug 04 09:09:31.755137 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.753922    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jvw5" event={"ID":"a409abeb-07c9-43f4-af5e-f9590d532649","Type":"ContainerStarted","Data":"6416a39e78c3deb42356738a36ce62ce81922cbe6351210cb43ecb57245483ab"}
Aug 04 09:09:31.854908 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.854835    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6jvw5" podStartSLOduration=3.14243584 podStartE2EDuration="4.854812695s" totalImagesPullingTime="1.602694806s" totalInitContainerRuntime="109.682049ms" isStatefulPod=true podCreationTimestamp="2026-08-04 09:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:31.777310834 +0000 UTC m=+103.035267889" watchObservedRunningTime="2026-08-04 09:09:31.854812695 +0000 UTC m=+103.112769750"
Aug 04 09:09:31.855417 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:31.855400    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f8f695f98-pcpbc"]
Aug 04 09:09:31.858013 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:09:31.857988    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50757688_799c_4ce8_9781_f454b95b74c7.slice/crio-f7682e2a613abe22bf47971c22847f69ff7ad74ba20c125129c82aedd184dfd3 WatchSource:0}: Error finding container f7682e2a613abe22bf47971c22847f69ff7ad74ba20c125129c82aedd184dfd3: Status 404 returned error can't find the container with id f7682e2a613abe22bf47971c22847f69ff7ad74ba20c125129c82aedd184dfd3
Aug 04 09:09:32.760145 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:32.760107    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc" event={"ID":"50757688-799c-4ce8-9781-f454b95b74c7","Type":"ContainerStarted","Data":"f7682e2a613abe22bf47971c22847f69ff7ad74ba20c125129c82aedd184dfd3"}
Aug 04 09:09:33.279550 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.279499    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 09:09:33.304562 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.304527    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.307337 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.307143    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 04 09:09:33.307337 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.307186    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 04 09:09:33.308138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.308113    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 04 09:09:33.308249 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.308194    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 04 09:09:33.308459 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.308441    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 04 09:09:33.308661 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.308629    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 04 09:09:33.308893 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.308445    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vcxw5\""
Aug 04 09:09:33.309580 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.309559    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 04 09:09:33.311762 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.311569    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 04 09:09:33.313438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.311973    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 04 09:09:33.313438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.312112    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 04 09:09:33.313438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.312231    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 04 09:09:33.313438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.312709    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 09:09:33.313438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.312949    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dv56igr155nmh\""
Aug 04 09:09:33.313438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.313157    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 04 09:09:33.313438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.313342    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 04 09:09:33.313854 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.313485    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 04 09:09:33.480554 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480508    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480554 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480613    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60353782-3641-4f05-a724-d6549148d5a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480667    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480689    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60353782-3641-4f05-a724-d6549148d5a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480720    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d5l8\" (UniqueName: \"kubernetes.io/projected/60353782-3641-4f05-a724-d6549148d5a5-kube-api-access-4d5l8\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480849    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480865    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.480891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.481258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480908    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.481258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-config\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.481258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.480970    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.481258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.481004    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.481258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.481071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.481258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.481093    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.481258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.481134    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.582529 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582493    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60353782-3641-4f05-a724-d6549148d5a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.582712 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.582712 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582571    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.582712 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582614    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60353782-3641-4f05-a724-d6549148d5a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.582712 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582644    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.582712 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582671    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.582712 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582696    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d5l8\" (UniqueName: \"kubernetes.io/projected/60353782-3641-4f05-a724-d6549148d5a5-kube-api-access-4d5l8\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582766    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582797    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582823    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-config\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.582986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.583017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583497 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.583058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583497 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.583086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583497 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.583108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.583905 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.583874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.584657 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.584525    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.587846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.585436    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.587846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.585801    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.587846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.586140    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.587846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.586962    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.587846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.587659    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.588158 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.588006    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60353782-3641-4f05-a724-d6549148d5a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.588214 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.588157    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60353782-3641-4f05-a724-d6549148d5a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.588420 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.588373    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60353782-3641-4f05-a724-d6549148d5a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.588986 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.588956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.589124 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.589086    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.590055 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.590028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.590150 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.590087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-config\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.590212 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.590173    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.590444 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.590418    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.591281 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.591262    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.591372 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.591295    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.591485 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.591466    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60353782-3641-4f05-a724-d6549148d5a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.595690 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.595665    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d5l8\" (UniqueName: \"kubernetes.io/projected/60353782-3641-4f05-a724-d6549148d5a5-kube-api-access-4d5l8\") pod \"prometheus-k8s-0\" (UID: \"60353782-3641-4f05-a724-d6549148d5a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.624584 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.624558    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:33.818294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:33.818263    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 09:09:33.873137 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:09:33.873077    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60353782_3641_4f05_a724_d6549148d5a5.slice/crio-aa58782744733fbe02f9eb88d2e3e25c6458440893a520c787ccea361ac8d41e WatchSource:0}: Error finding container aa58782744733fbe02f9eb88d2e3e25c6458440893a520c787ccea361ac8d41e: Status 404 returned error can't find the container with id aa58782744733fbe02f9eb88d2e3e25c6458440893a520c787ccea361ac8d41e
Aug 04 09:09:34.773179 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:34.772749    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc" event={"ID":"50757688-799c-4ce8-9781-f454b95b74c7","Type":"ContainerStarted","Data":"2cde5f79602b954ccbbb2fdf2e1b598606e01ec7ad71ed6d5e9e9a8cbb4ff6b0"}
Aug 04 09:09:34.774405 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:34.774379    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerStarted","Data":"aa58782744733fbe02f9eb88d2e3e25c6458440893a520c787ccea361ac8d41e"}
Aug 04 09:09:35.779669 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:35.779637    2558 generic.go:356] "Generic (PLEG): container finished" podID="60353782-3641-4f05-a724-d6549148d5a5" containerID="4e2948d1b1826b7a35b878f5077242716b07ad0bd94920b221c2d6b2995bde35" exitCode=0
Aug 04 09:09:35.780073 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:35.779728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerDied","Data":"4e2948d1b1826b7a35b878f5077242716b07ad0bd94920b221c2d6b2995bde35"}
Aug 04 09:09:35.810483 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:35.810344    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc" podStartSLOduration=2.774291521 podStartE2EDuration="4.810327786s" totalImagesPullingTime="2.036036265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:34.792048248 +0000 UTC m=+106.050005302" watchObservedRunningTime="2026-08-04 09:09:35.810327786 +0000 UTC m=+107.068284841"
Aug 04 09:09:38.661878 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:38.661840    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l9zs2"
Aug 04 09:09:39.199132 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:39.199104    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t9sl8_47752d1b-8f63-439d-b354-532f3b7ed228/serve-healthcheck-canary/0.log"
Aug 04 09:09:39.798206 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:39.798172    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerStarted","Data":"c8cfc582198b0332f83a25f506c8f6185b60975e77018765165a6341206ff759"}
Aug 04 09:09:40.803569 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:40.803537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerStarted","Data":"b2dc6c1106c75e7a761173c630b615c069f259d635385ff4cd332211723a47ea"}
Aug 04 09:09:42.814890 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:42.814805    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerStarted","Data":"b85dd45dcb394b0c246f8939d5652d1908763ab2b586ac45feed0ec2205abcea"}
Aug 04 09:09:42.814890 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:42.814874    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerStarted","Data":"eb8d7ccdc1d46511c8aa797446b7c2e77f4df700bf359431dc4de176428e45bd"}
Aug 04 09:09:42.815250 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:42.814906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerStarted","Data":"3a03cdf5d07aba354ab42e3881082d4eba4a548740a76f7684647eceae8aecfc"}
Aug 04 09:09:42.815250 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:42.814935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60353782-3641-4f05-a724-d6549148d5a5","Type":"ContainerStarted","Data":"55a070bab7335b4d1b9bf78f2930e2557551439e5c8a83a33c8e0c84f05502c1"}
Aug 04 09:09:42.845689 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:42.845633    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.1480376039999998 podStartE2EDuration="9.845615669s" totalImagesPullingTime="7.681954615s" totalInitContainerRuntime="15.62345ms" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:42.844038536 +0000 UTC m=+114.101995602" watchObservedRunningTime="2026-08-04 09:09:42.845615669 +0000 UTC m=+114.103572718"
Aug 04 09:09:43.625245 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:43.625211    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:09:51.674094 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:51.674058    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:09:51.674585 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:09:51.674132    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:10:11.678822 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:10:11.678792    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:10:11.682661 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:10:11.682638    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f8f695f98-pcpbc"
Aug 04 09:10:33.625098 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:10:33.625058    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:10:33.643895 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:10:33.643867    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:10:34.010444 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:10:34.010369    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 09:12:49.172042 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:12:49.172012    2558 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 09:13:21.708700 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.708667    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"]
Aug 04 09:13:21.712016 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.712000    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.714522 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.714503    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-52t6g\""
Aug 04 09:13:21.714634 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.714604    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 04 09:13:21.714675 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.714639    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 04 09:13:21.714915 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.714898    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 04 09:13:21.715209 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.715196    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 04 09:13:21.732493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.732473    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"]
Aug 04 09:13:21.839030 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.838996    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.839157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.839035    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-tmp\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.839157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.839063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.839157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.839092    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-manifests\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.839157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.839131    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stjhj\" (UniqueName: \"kubernetes.io/projected/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-kube-api-access-stjhj\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.939903 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.939857    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.940059 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.939910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-tmp\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.940059 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.940029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.940176 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.940069    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-manifests\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.940176 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.940150    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stjhj\" (UniqueName: \"kubernetes.io/projected/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-kube-api-access-stjhj\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.940487 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.940469    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-manifests\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.942358 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.942338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-tmp\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.942515 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.942496    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.942563 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.942528    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:21.952185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:21.952159    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stjhj\" (UniqueName: \"kubernetes.io/projected/afc0c3dc-f1d0-4f57-9765-b8e750bc8b42-kube-api-access-stjhj\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-plpvv\" (UID: \"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:22.021865 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:22.021802    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:22.161064 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:22.161016    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"]
Aug 04 09:13:22.163449 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:13:22.163419    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc0c3dc_f1d0_4f57_9765_b8e750bc8b42.slice/crio-29c3c7a23d822af829dfe98bc034f530e382334409a669ce5000caba4723fec0 WatchSource:0}: Error finding container 29c3c7a23d822af829dfe98bc034f530e382334409a669ce5000caba4723fec0: Status 404 returned error can't find the container with id 29c3c7a23d822af829dfe98bc034f530e382334409a669ce5000caba4723fec0
Aug 04 09:13:22.165171 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:22.165154    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 09:13:22.546275 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:22.546246    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv" event={"ID":"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42","Type":"ContainerStarted","Data":"29c3c7a23d822af829dfe98bc034f530e382334409a669ce5000caba4723fec0"}
Aug 04 09:13:25.560452 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:25.560373    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv" event={"ID":"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42","Type":"ContainerStarted","Data":"2f9f954667f2ce405ee419932aaa118fcbe5b4eda5398259ca0ef78ab71ee9c9"}
Aug 04 09:13:26.565338 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:26.565256    2558 generic.go:356] "Generic (PLEG): container finished" podID="afc0c3dc-f1d0-4f57-9765-b8e750bc8b42" containerID="2f9f954667f2ce405ee419932aaa118fcbe5b4eda5398259ca0ef78ab71ee9c9" exitCode=0
Aug 04 09:13:26.565338 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:26.565322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv" event={"ID":"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42","Type":"ContainerDied","Data":"2f9f954667f2ce405ee419932aaa118fcbe5b4eda5398259ca0ef78ab71ee9c9"}
Aug 04 09:13:27.571582 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:27.571535    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv" event={"ID":"afc0c3dc-f1d0-4f57-9765-b8e750bc8b42","Type":"ContainerStarted","Data":"4a7caaeef078c3bb983d41614b361a820625c5bda53c59c6e84ef004ef9fa9ad"}
Aug 04 09:13:27.572066 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:27.571683    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:27.597032 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:27.596985    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv" podStartSLOduration=2.166484663 podStartE2EDuration="6.596970957s" totalImagesPullingTime="3.410930622s" totalInitContainerRuntime="1.019555672s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:27.595648048 +0000 UTC m=+338.853605116" watchObservedRunningTime="2026-08-04 09:13:27.596970957 +0000 UTC m=+338.854928011"
Aug 04 09:13:38.577075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:38.577042    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-plpvv"
Aug 04 09:13:42.060717 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.060639    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"]
Aug 04 09:13:42.067390 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.067359    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.071642 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.071616    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 04 09:13:42.071806 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.071616    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 04 09:13:42.072558 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.072534    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j5xst\""
Aug 04 09:13:42.072662 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.072541    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 04 09:13:42.072662 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.072586    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 04 09:13:42.084442 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.084421    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"]
Aug 04 09:13:42.098219 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.098196    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07143299-3a85-4155-ab9d-5c3998986c39-tmp\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.098331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.098244    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85hpj\" (UniqueName: \"kubernetes.io/projected/07143299-3a85-4155-ab9d-5c3998986c39-kube-api-access-85hpj\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.098331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.098273    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07143299-3a85-4155-ab9d-5c3998986c39-tls-certs\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.198987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.198940    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07143299-3a85-4155-ab9d-5c3998986c39-tls-certs\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.199169 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.199014    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07143299-3a85-4155-ab9d-5c3998986c39-tmp\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.199169 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.199089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85hpj\" (UniqueName: \"kubernetes.io/projected/07143299-3a85-4155-ab9d-5c3998986c39-kube-api-access-85hpj\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.201699 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.201676    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07143299-3a85-4155-ab9d-5c3998986c39-tmp\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.202097 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.202074    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07143299-3a85-4155-ab9d-5c3998986c39-tls-certs\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.217168 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.217137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85hpj\" (UniqueName: \"kubernetes.io/projected/07143299-3a85-4155-ab9d-5c3998986c39-kube-api-access-85hpj\") pod \"kube-auth-proxy-6bcc56ccff-9rjg7\" (UID: \"07143299-3a85-4155-ab9d-5c3998986c39\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.378543 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.378507    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"
Aug 04 09:13:42.521464 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.521393    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7"]
Aug 04 09:13:42.523928 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:13:42.523899    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07143299_3a85_4155_ab9d_5c3998986c39.slice/crio-8b9ad375a5d3d7c75e865e6cdc00872c746930d506edd58b68c007201b35dc01 WatchSource:0}: Error finding container 8b9ad375a5d3d7c75e865e6cdc00872c746930d506edd58b68c007201b35dc01: Status 404 returned error can't find the container with id 8b9ad375a5d3d7c75e865e6cdc00872c746930d506edd58b68c007201b35dc01
Aug 04 09:13:42.625046 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:42.625006    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7" event={"ID":"07143299-3a85-4155-ab9d-5c3998986c39","Type":"ContainerStarted","Data":"8b9ad375a5d3d7c75e865e6cdc00872c746930d506edd58b68c007201b35dc01"}
Aug 04 09:13:43.588272 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.587652    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"]
Aug 04 09:13:43.591320 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.591289    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.594593 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.594569    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 04 09:13:43.594910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.594872    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 04 09:13:43.595074 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.595039    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 04 09:13:43.595164 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.595101    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8qlk7\""
Aug 04 09:13:43.595233 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.595177    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 04 09:13:43.609803 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.609682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.609803 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.609720    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92cp8\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-kube-api-access-92cp8\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.609994 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.609832    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.609994 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.609898    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.609994 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.609932    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ba57-95dc-4442-8546-8467c5f89ce3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.609994 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.609957    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.610204 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.610029    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.611703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.611677    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"]
Aug 04 09:13:43.710664 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.710625    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.710857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.710672    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92cp8\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-kube-api-access-92cp8\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.710857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.710714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.710857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.710778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.710857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.710810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ba57-95dc-4442-8546-8467c5f89ce3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.710857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.710836    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.711126 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.710878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.712419 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.712371    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.713956 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.713902    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ba57-95dc-4442-8546-8467c5f89ce3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.714264 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.714235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.715342 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.715314    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.715667 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.715645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.720799 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.720774    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.721368 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.721346    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92cp8\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-kube-api-access-92cp8\") pod \"istiod-openshift-gateway-7b7995d5cd-rsg99\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:43.905323 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:43.905236    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:44.072710 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:44.072161    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"]
Aug 04 09:13:44.075111 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:13:44.075079    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0457ba57_95dc_4442_8546_8467c5f89ce3.slice/crio-c3b345e53a2dcdcc615ae652a721b1ff18ca7871b7dd9aa7ea73f41147831f19 WatchSource:0}: Error finding container c3b345e53a2dcdcc615ae652a721b1ff18ca7871b7dd9aa7ea73f41147831f19: Status 404 returned error can't find the container with id c3b345e53a2dcdcc615ae652a721b1ff18ca7871b7dd9aa7ea73f41147831f19
Aug 04 09:13:44.637418 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:44.637380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99" event={"ID":"0457ba57-95dc-4442-8546-8467c5f89ce3","Type":"ContainerStarted","Data":"c3b345e53a2dcdcc615ae652a721b1ff18ca7871b7dd9aa7ea73f41147831f19"}
Aug 04 09:13:46.966705 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:46.966666    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:13:46.967116 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:46.966785    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:13:47.655258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:47.655204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7" event={"ID":"07143299-3a85-4155-ab9d-5c3998986c39","Type":"ContainerStarted","Data":"5ba84bd4e6aeb75f0a9b246338c50a6338bad87850a0ed8ed1842d49b140d7ec"}
Aug 04 09:13:47.657402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:47.657367    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99" event={"ID":"0457ba57-95dc-4442-8546-8467c5f89ce3","Type":"ContainerStarted","Data":"3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a"}
Aug 04 09:13:47.657979 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:47.657957    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:47.674972 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:47.674907    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-9rjg7" podStartSLOduration=1.279249422 podStartE2EDuration="5.674884979s" totalImagesPullingTime="4.395635557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:47.673493931 +0000 UTC m=+358.931450988" watchObservedRunningTime="2026-08-04 09:13:47.674884979 +0000 UTC m=+358.932842036"
Aug 04 09:13:47.698090 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:47.698018    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99" podStartSLOduration=1.809512762 podStartE2EDuration="4.697994177s" totalImagesPullingTime="2.888481415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:47.696969083 +0000 UTC m=+358.954926138" watchObservedRunningTime="2026-08-04 09:13:47.697994177 +0000 UTC m=+358.955951230"
Aug 04 09:13:48.662502 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:48.662475    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:52.121273 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.121238    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"]
Aug 04 09:13:52.126053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.126033    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.139665 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.139642    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"]
Aug 04 09:13:52.189422 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.189392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1442d06-baac-4b71-8db0-def11ce5f8ae-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.189550 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.189428    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.189550 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.189487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.189550 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.189520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2mhm\" (UniqueName: \"kubernetes.io/projected/a1442d06-baac-4b71-8db0-def11ce5f8ae-kube-api-access-f2mhm\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.189650 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.189590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.189696 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.189674    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.189756 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.189711    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.290772 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.290716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.290910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.290778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.290910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.290824    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1442d06-baac-4b71-8db0-def11ce5f8ae-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.290910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.290853    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.290910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.290906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.291117 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.290959    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2mhm\" (UniqueName: \"kubernetes.io/projected/a1442d06-baac-4b71-8db0-def11ce5f8ae-kube-api-access-f2mhm\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.291117 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.290986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.291678 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.291657    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.293357 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.293315    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.293469 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.293407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.293545 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.293476    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1442d06-baac-4b71-8db0-def11ce5f8ae-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.293674 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.293655    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1442d06-baac-4b71-8db0-def11ce5f8ae-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.298649 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.298626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2mhm\" (UniqueName: \"kubernetes.io/projected/a1442d06-baac-4b71-8db0-def11ce5f8ae-kube-api-access-f2mhm\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.298758 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.298649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1442d06-baac-4b71-8db0-def11ce5f8ae-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rjr8s\" (UID: \"a1442d06-baac-4b71-8db0-def11ce5f8ae\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.435110 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.435049    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.585825 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:13:52.585782    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1442d06_baac_4b71_8db0_def11ce5f8ae.slice/crio-40790a4fc400a7d5f2b1d847d0d811b1c136efe21fc4b731b9168f5d06ff712d WatchSource:0}: Error finding container 40790a4fc400a7d5f2b1d847d0d811b1c136efe21fc4b731b9168f5d06ff712d: Status 404 returned error can't find the container with id 40790a4fc400a7d5f2b1d847d0d811b1c136efe21fc4b731b9168f5d06ff712d
Aug 04 09:13:52.586133 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.586111    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"]
Aug 04 09:13:52.588091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.588060    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:13:52.588161 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.588125    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:13:52.677649 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.677611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s" event={"ID":"a1442d06-baac-4b71-8db0-def11ce5f8ae","Type":"ContainerStarted","Data":"7a62ca269e1bb99dfc36222b7c3204da20046babc7afcc79cab278be848e3927"}
Aug 04 09:13:52.677837 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.677671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s" event={"ID":"a1442d06-baac-4b71-8db0-def11ce5f8ae","Type":"ContainerStarted","Data":"40790a4fc400a7d5f2b1d847d0d811b1c136efe21fc4b731b9168f5d06ff712d"}
Aug 04 09:13:52.677922 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.677861    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:52.709677 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:52.709563    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s" podStartSLOduration=0.709539342 podStartE2EDuration="709.539342ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:52.706036608 +0000 UTC m=+363.963993661" watchObservedRunningTime="2026-08-04 09:13:52.709539342 +0000 UTC m=+363.967496405"
Aug 04 09:13:53.683963 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:53.683935    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rjr8s"
Aug 04 09:13:53.796338 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:53.796303    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"]
Aug 04 09:13:53.796584 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:53.796556    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99" podUID="0457ba57-95dc-4442-8546-8467c5f89ce3" containerName="discovery" containerID="cri-o://3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a" gracePeriod=30
Aug 04 09:13:54.038224 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.038197    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:54.108009 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.107980    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-token\") pod \"0457ba57-95dc-4442-8546-8467c5f89ce3\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") "
Aug 04 09:13:54.108155 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.108014    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-cacerts\") pod \"0457ba57-95dc-4442-8546-8467c5f89ce3\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") "
Aug 04 09:13:54.108155 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.108032    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-ca-configmap\") pod \"0457ba57-95dc-4442-8546-8467c5f89ce3\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") "
Aug 04 09:13:54.108155 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.108068    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ba57-95dc-4442-8546-8467c5f89ce3-local-certs\") pod \"0457ba57-95dc-4442-8546-8467c5f89ce3\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") "
Aug 04 09:13:54.108155 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.108092    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92cp8\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-kube-api-access-92cp8\") pod \"0457ba57-95dc-4442-8546-8467c5f89ce3\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") "
Aug 04 09:13:54.108155 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.108124    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-kubeconfig\") pod \"0457ba57-95dc-4442-8546-8467c5f89ce3\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") "
Aug 04 09:13:54.108408 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.108221    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-dns-cert\") pod \"0457ba57-95dc-4442-8546-8467c5f89ce3\" (UID: \"0457ba57-95dc-4442-8546-8467c5f89ce3\") "
Aug 04 09:13:54.108469 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.108429    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0457ba57-95dc-4442-8546-8467c5f89ce3" (UID: "0457ba57-95dc-4442-8546-8467c5f89ce3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:13:54.110816 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.110776    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-cacerts" (OuterVolumeSpecName: "cacerts") pod "0457ba57-95dc-4442-8546-8467c5f89ce3" (UID: "0457ba57-95dc-4442-8546-8467c5f89ce3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:13:54.110816 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.110780    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0457ba57-95dc-4442-8546-8467c5f89ce3-local-certs" (OuterVolumeSpecName: "local-certs") pod "0457ba57-95dc-4442-8546-8467c5f89ce3" (UID: "0457ba57-95dc-4442-8546-8467c5f89ce3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:13:54.110816 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.110781    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-token" (OuterVolumeSpecName: "istio-token") pod "0457ba57-95dc-4442-8546-8467c5f89ce3" (UID: "0457ba57-95dc-4442-8546-8467c5f89ce3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:13:54.111048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.110949    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-kube-api-access-92cp8" (OuterVolumeSpecName: "kube-api-access-92cp8") pod "0457ba57-95dc-4442-8546-8467c5f89ce3" (UID: "0457ba57-95dc-4442-8546-8467c5f89ce3"). InnerVolumeSpecName "kube-api-access-92cp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:13:54.111048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.110993    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0457ba57-95dc-4442-8546-8467c5f89ce3" (UID: "0457ba57-95dc-4442-8546-8467c5f89ce3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:13:54.111150 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.111065    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0457ba57-95dc-4442-8546-8467c5f89ce3" (UID: "0457ba57-95dc-4442-8546-8467c5f89ce3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:13:54.209674 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.209640    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-dns-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:13:54.209674 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.209668    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-token\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:13:54.209674 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.209679    2558 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-cacerts\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:13:54.209898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.209689    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-csr-ca-configmap\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:13:54.209898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.209699    2558 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0457ba57-95dc-4442-8546-8467c5f89ce3-local-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:13:54.209898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.209708    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92cp8\" (UniqueName: \"kubernetes.io/projected/0457ba57-95dc-4442-8546-8467c5f89ce3-kube-api-access-92cp8\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:13:54.209898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.209717    2558 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0457ba57-95dc-4442-8546-8467c5f89ce3-istio-kubeconfig\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:13:54.688559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.688520    2558 generic.go:356] "Generic (PLEG): container finished" podID="0457ba57-95dc-4442-8546-8467c5f89ce3" containerID="3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a" exitCode=0
Aug 04 09:13:54.689021 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.688568    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"
Aug 04 09:13:54.689021 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.688629    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99" event={"ID":"0457ba57-95dc-4442-8546-8467c5f89ce3","Type":"ContainerDied","Data":"3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a"}
Aug 04 09:13:54.689021 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.688674    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99" event={"ID":"0457ba57-95dc-4442-8546-8467c5f89ce3","Type":"ContainerDied","Data":"c3b345e53a2dcdcc615ae652a721b1ff18ca7871b7dd9aa7ea73f41147831f19"}
Aug 04 09:13:54.689021 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.688694    2558 scope.go:117] "RemoveContainer" containerID="3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a"
Aug 04 09:13:54.697307 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.697289    2558 scope.go:117] "RemoveContainer" containerID="3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a"
Aug 04 09:13:54.697554 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:13:54.697536    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a\": container with ID starting with 3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a not found: ID does not exist" containerID="3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a"
Aug 04 09:13:54.697616 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.697570    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a"} err="failed to get container status \"3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a\": rpc error: code = NotFound desc = could not find container \"3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a\": container with ID starting with 3988d03ba63a707eca2c05898807b33c6daf531beaa3100a23671938fe72b41a not found: ID does not exist"
Aug 04 09:13:54.712497 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.712467    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"]
Aug 04 09:13:54.719629 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:54.719607    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rsg99"]
Aug 04 09:13:55.298591 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:13:55.296855    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0457ba57-95dc-4442-8546-8467c5f89ce3" path="/var/lib/kubelet/pods/0457ba57-95dc-4442-8546-8467c5f89ce3/volumes"
Aug 04 09:14:57.903876 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.903837    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"]
Aug 04 09:14:57.904313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.904148    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0457ba57-95dc-4442-8546-8467c5f89ce3" containerName="discovery"
Aug 04 09:14:57.904313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.904159    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0457ba57-95dc-4442-8546-8467c5f89ce3" containerName="discovery"
Aug 04 09:14:57.904313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.904215    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0457ba57-95dc-4442-8546-8467c5f89ce3" containerName="discovery"
Aug 04 09:14:57.907966 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.907950    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"
Aug 04 09:14:57.911280 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.911244    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Aug 04 09:14:57.911406 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.911277    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 04 09:14:57.911406 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.911308    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 04 09:14:57.912216 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.912197    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6z9tq\""
Aug 04 09:14:57.918079 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.918059    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"]
Aug 04 09:14:57.996105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:57.996080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s8xd\" (UniqueName: \"kubernetes.io/projected/89a290fa-fa2e-439f-a91c-03e06b0074c2-kube-api-access-5s8xd\") pod \"dns-operator-controller-manager-5985564776-wbctw\" (UID: \"89a290fa-fa2e-439f-a91c-03e06b0074c2\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"
Aug 04 09:14:58.096791 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:58.096763    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s8xd\" (UniqueName: \"kubernetes.io/projected/89a290fa-fa2e-439f-a91c-03e06b0074c2-kube-api-access-5s8xd\") pod \"dns-operator-controller-manager-5985564776-wbctw\" (UID: \"89a290fa-fa2e-439f-a91c-03e06b0074c2\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"
Aug 04 09:14:58.108387 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:58.108359    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s8xd\" (UniqueName: \"kubernetes.io/projected/89a290fa-fa2e-439f-a91c-03e06b0074c2-kube-api-access-5s8xd\") pod \"dns-operator-controller-manager-5985564776-wbctw\" (UID: \"89a290fa-fa2e-439f-a91c-03e06b0074c2\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"
Aug 04 09:14:58.218543 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:58.218475    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"
Aug 04 09:14:58.347565 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:58.347532    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"]
Aug 04 09:14:58.350532 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:14:58.350501    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a290fa_fa2e_439f_a91c_03e06b0074c2.slice/crio-7185bea0d59a551a0d0b6cbc767c0e6cc4cd6792d25d4b0e5775f3938c01cc51 WatchSource:0}: Error finding container 7185bea0d59a551a0d0b6cbc767c0e6cc4cd6792d25d4b0e5775f3938c01cc51: Status 404 returned error can't find the container with id 7185bea0d59a551a0d0b6cbc767c0e6cc4cd6792d25d4b0e5775f3938c01cc51
Aug 04 09:14:58.941179 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:14:58.941143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw" event={"ID":"89a290fa-fa2e-439f-a91c-03e06b0074c2","Type":"ContainerStarted","Data":"7185bea0d59a551a0d0b6cbc767c0e6cc4cd6792d25d4b0e5775f3938c01cc51"}
Aug 04 09:15:01.955965 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:01.955923    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw" event={"ID":"89a290fa-fa2e-439f-a91c-03e06b0074c2","Type":"ContainerStarted","Data":"34026c08ad98aabee1ce1fbc4af4d01f1a47b5c3dd75ee2bab2920b60f10db4e"}
Aug 04 09:15:01.956397 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:01.956060    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"
Aug 04 09:15:01.984794 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:01.984724    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw" podStartSLOduration=2.398582961 podStartE2EDuration="4.984710548s" totalImagesPullingTime="2.586127587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:15:01.984497527 +0000 UTC m=+433.242454580" watchObservedRunningTime="2026-08-04 09:15:01.984710548 +0000 UTC m=+433.242667601"
Aug 04 09:15:02.396002 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.395970    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"]
Aug 04 09:15:02.399228 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.399211    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.402460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.402437    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9rh2j\""
Aug 04 09:15:02.411161 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.411138    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"]
Aug 04 09:15:02.436350 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.436330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfk6h\" (UniqueName: \"kubernetes.io/projected/c81d9d93-6822-4884-836d-65a04625e4d4-kube-api-access-pfk6h\") pod \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.436470 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.436375    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81d9d93-6822-4884-836d-65a04625e4d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.537436 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.537409    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfk6h\" (UniqueName: \"kubernetes.io/projected/c81d9d93-6822-4884-836d-65a04625e4d4-kube-api-access-pfk6h\") pod \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.537566 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.537466    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81d9d93-6822-4884-836d-65a04625e4d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.537839 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.537823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81d9d93-6822-4884-836d-65a04625e4d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.555982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.555959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfk6h\" (UniqueName: \"kubernetes.io/projected/c81d9d93-6822-4884-836d-65a04625e4d4-kube-api-access-pfk6h\") pod \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.709813 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.709724    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:02.848455 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.848427    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"]
Aug 04 09:15:02.851825 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:15:02.851798    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81d9d93_6822_4884_836d_65a04625e4d4.slice/crio-5bd5a7eb9a15b5a538c935a9f8ea8a24b2580ce1fb85f7bc29c21bf60747ccc8 WatchSource:0}: Error finding container 5bd5a7eb9a15b5a538c935a9f8ea8a24b2580ce1fb85f7bc29c21bf60747ccc8: Status 404 returned error can't find the container with id 5bd5a7eb9a15b5a538c935a9f8ea8a24b2580ce1fb85f7bc29c21bf60747ccc8
Aug 04 09:15:02.961240 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:02.961150    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v" event={"ID":"c81d9d93-6822-4884-836d-65a04625e4d4","Type":"ContainerStarted","Data":"5bd5a7eb9a15b5a538c935a9f8ea8a24b2580ce1fb85f7bc29c21bf60747ccc8"}
Aug 04 09:15:05.089274 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.089237    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-8sqfh"]
Aug 04 09:15:05.092908 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.092886    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh"
Aug 04 09:15:05.095572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.095546    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nfdtf\""
Aug 04 09:15:05.104908 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.104850    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-8sqfh"]
Aug 04 09:15:05.163020 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.162988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69c9r\" (UniqueName: \"kubernetes.io/projected/ddaa39b9-0a08-4b34-bc82-a576086cf983-kube-api-access-69c9r\") pod \"authorino-operator-68d88948d8-8sqfh\" (UID: \"ddaa39b9-0a08-4b34-bc82-a576086cf983\") " pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh"
Aug 04 09:15:05.263937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.263897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69c9r\" (UniqueName: \"kubernetes.io/projected/ddaa39b9-0a08-4b34-bc82-a576086cf983-kube-api-access-69c9r\") pod \"authorino-operator-68d88948d8-8sqfh\" (UID: \"ddaa39b9-0a08-4b34-bc82-a576086cf983\") " pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh"
Aug 04 09:15:05.275853 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.275824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69c9r\" (UniqueName: \"kubernetes.io/projected/ddaa39b9-0a08-4b34-bc82-a576086cf983-kube-api-access-69c9r\") pod \"authorino-operator-68d88948d8-8sqfh\" (UID: \"ddaa39b9-0a08-4b34-bc82-a576086cf983\") " pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh"
Aug 04 09:15:05.405858 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.405779    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh"
Aug 04 09:15:05.544821 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:05.544796    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-8sqfh"]
Aug 04 09:15:06.669677 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:15:06.669639    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddaa39b9_0a08_4b34_bc82_a576086cf983.slice/crio-9fe2e613c26050e46856c1939211b4f270123162829a7529784f6d01ffa77ab4 WatchSource:0}: Error finding container 9fe2e613c26050e46856c1939211b4f270123162829a7529784f6d01ffa77ab4: Status 404 returned error can't find the container with id 9fe2e613c26050e46856c1939211b4f270123162829a7529784f6d01ffa77ab4
Aug 04 09:15:06.980228 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:06.980140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh" event={"ID":"ddaa39b9-0a08-4b34-bc82-a576086cf983","Type":"ContainerStarted","Data":"9fe2e613c26050e46856c1939211b4f270123162829a7529784f6d01ffa77ab4"}
Aug 04 09:15:06.981781 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:06.981755    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v" event={"ID":"c81d9d93-6822-4884-836d-65a04625e4d4","Type":"ContainerStarted","Data":"3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c"}
Aug 04 09:15:06.981901 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:06.981817    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:07.003687 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.003644    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v" podStartSLOduration=1.119788009 podStartE2EDuration="5.003627234s" totalImagesPullingTime="3.883839225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:15:07.001628885 +0000 UTC m=+438.259585949" watchObservedRunningTime="2026-08-04 09:15:07.003627234 +0000 UTC m=+438.261584287"
Aug 04 09:15:07.087443 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.087412    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"]
Aug 04 09:15:07.090703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.090681    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:07.093146 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.093128    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-p4wbs\""
Aug 04 09:15:07.103820 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.103785    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"]
Aug 04 09:15:07.180528 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.180491    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrrfx\" (UniqueName: \"kubernetes.io/projected/39c42666-6e6f-4311-b8d3-082f8fae288d-kube-api-access-hrrfx\") pod \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" (UID: \"39c42666-6e6f-4311-b8d3-082f8fae288d\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:07.281713 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.281626    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrrfx\" (UniqueName: \"kubernetes.io/projected/39c42666-6e6f-4311-b8d3-082f8fae288d-kube-api-access-hrrfx\") pod \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" (UID: \"39c42666-6e6f-4311-b8d3-082f8fae288d\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:07.293760 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.293702    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrrfx\" (UniqueName: \"kubernetes.io/projected/39c42666-6e6f-4311-b8d3-082f8fae288d-kube-api-access-hrrfx\") pod \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" (UID: \"39c42666-6e6f-4311-b8d3-082f8fae288d\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:07.400262 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.400223    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:07.557037 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.557006    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"]
Aug 04 09:15:07.559692 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:15:07.559665    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39c42666_6e6f_4311_b8d3_082f8fae288d.slice/crio-4a7f12330354b127549ca07deac34d87df77662f8e40d527c3b1be102555a438 WatchSource:0}: Error finding container 4a7f12330354b127549ca07deac34d87df77662f8e40d527c3b1be102555a438: Status 404 returned error can't find the container with id 4a7f12330354b127549ca07deac34d87df77662f8e40d527c3b1be102555a438
Aug 04 09:15:07.988103 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:07.988070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl" event={"ID":"39c42666-6e6f-4311-b8d3-082f8fae288d","Type":"ContainerStarted","Data":"4a7f12330354b127549ca07deac34d87df77662f8e40d527c3b1be102555a438"}
Aug 04 09:15:08.994804 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:08.994765    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh" event={"ID":"ddaa39b9-0a08-4b34-bc82-a576086cf983","Type":"ContainerStarted","Data":"96dfa5593da2a205cd786b604e49f568baf40eeb68e6c62198e6e7c179648cbe"}
Aug 04 09:15:08.995219 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:08.994965    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh"
Aug 04 09:15:09.012056 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:09.012002    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh" podStartSLOduration=2.709634211 podStartE2EDuration="4.011982329s" totalImagesPullingTime="1.302348118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:15:09.011097238 +0000 UTC m=+440.269054293" watchObservedRunningTime="2026-08-04 09:15:09.011982329 +0000 UTC m=+440.269939384"
Aug 04 09:15:10.000841 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:10.000804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl" event={"ID":"39c42666-6e6f-4311-b8d3-082f8fae288d","Type":"ContainerStarted","Data":"e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2"}
Aug 04 09:15:10.001296 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:10.001052    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:10.020033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:10.019975    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl" podStartSLOduration=1.417324477 podStartE2EDuration="3.019960291s" totalImagesPullingTime="1.602635814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:15:10.01840696 +0000 UTC m=+441.276364039" watchObservedRunningTime="2026-08-04 09:15:10.019960291 +0000 UTC m=+441.277917345"
Aug 04 09:15:12.963904 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:12.963870    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-wbctw"
Aug 04 09:15:17.990262 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:17.990229    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.003147 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.003117    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-8sqfh"
Aug 04 09:15:20.113010 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.112979    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"]
Aug 04 09:15:20.113201 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.113166    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" containerName="manager" containerID="cri-o://3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c" gracePeriod=2
Aug 04 09:15:20.119066 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.119030    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"]
Aug 04 09:15:20.131914 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.131878    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"]
Aug 04 09:15:20.132287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.132126    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" containerName="manager" containerID="cri-o://e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2" gracePeriod=2
Aug 04 09:15:20.134425 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.134402    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:20.134779 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.134710    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.140417 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.140387    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"]
Aug 04 09:15:20.141369 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.141221    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" containerName="manager"
Aug 04 09:15:20.141369 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.141245    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" containerName="manager"
Aug 04 09:15:20.141369 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.141337    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" containerName="manager"
Aug 04 09:15:20.145668 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.145631    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:20.147399 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.147374    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.148990 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.148972    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"]
Aug 04 09:15:20.149085 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.149061    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.151183 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.151151    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.153051 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.153020    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:20.156507 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.156482    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"]
Aug 04 09:15:20.156962 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.156944    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" containerName="manager"
Aug 04 09:15:20.157019 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.156966    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" containerName="manager"
Aug 04 09:15:20.157051 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.157042    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" containerName="manager"
Aug 04 09:15:20.161403 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.161383    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"]
Aug 04 09:15:20.161502 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.161471    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"
Aug 04 09:15:20.172677 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.172652    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"]
Aug 04 09:15:20.185622 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.185598    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.205502 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.205473    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:20.298216 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.298130    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66505920-6d99-49fe-949b-2e064b56b667-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-x4ppw\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.298382 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.298298    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwv6s\" (UniqueName: \"kubernetes.io/projected/66505920-6d99-49fe-949b-2e064b56b667-kube-api-access-rwv6s\") pod \"kuadrant-operator-controller-manager-58fc56757c-x4ppw\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.298382 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.298349    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qpt6\" (UniqueName: \"kubernetes.io/projected/98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba-kube-api-access-5qpt6\") pod \"limitador-operator-controller-manager-6c549b6d85-zgsbv\" (UID: \"98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"
Aug 04 09:15:20.370833 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.370812    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:20.372991 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.372966    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:20.373594 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.373575    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.374940 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.374920    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.376568 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.376550    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:20.378421 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:20.378404    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:20.398773 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.398727    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwv6s\" (UniqueName: \"kubernetes.io/projected/66505920-6d99-49fe-949b-2e064b56b667-kube-api-access-rwv6s\") pod \"kuadrant-operator-controller-manager-58fc56757c-x4ppw\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.398856 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.398788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qpt6\" (UniqueName: \"kubernetes.io/projected/98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba-kube-api-access-5qpt6\") pod \"limitador-operator-controller-manager-6c549b6d85-zgsbv\" (UID: \"98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"
Aug 04 09:15:20.398856 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.398818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66505920-6d99-49fe-949b-2e064b56b667-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-x4ppw\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.399190 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.399170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66505920-6d99-49fe-949b-2e064b56b667-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-x4ppw\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.410198 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.410179    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwv6s\" (UniqueName: \"kubernetes.io/projected/66505920-6d99-49fe-949b-2e064b56b667-kube-api-access-rwv6s\") pod \"kuadrant-operator-controller-manager-58fc56757c-x4ppw\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.410274 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.410258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qpt6\" (UniqueName: \"kubernetes.io/projected/98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba-kube-api-access-5qpt6\") pod \"limitador-operator-controller-manager-6c549b6d85-zgsbv\" (UID: \"98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"
Aug 04 09:15:20.499859 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.499833    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfk6h\" (UniqueName: \"kubernetes.io/projected/c81d9d93-6822-4884-836d-65a04625e4d4-kube-api-access-pfk6h\") pod \"c81d9d93-6822-4884-836d-65a04625e4d4\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") "
Aug 04 09:15:20.499980 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.499873    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrrfx\" (UniqueName: \"kubernetes.io/projected/39c42666-6e6f-4311-b8d3-082f8fae288d-kube-api-access-hrrfx\") pod \"39c42666-6e6f-4311-b8d3-082f8fae288d\" (UID: \"39c42666-6e6f-4311-b8d3-082f8fae288d\") "
Aug 04 09:15:20.499980 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.499893    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81d9d93-6822-4884-836d-65a04625e4d4-extensions-socket-volume\") pod \"c81d9d93-6822-4884-836d-65a04625e4d4\" (UID: \"c81d9d93-6822-4884-836d-65a04625e4d4\") "
Aug 04 09:15:20.500425 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.500400    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c81d9d93-6822-4884-836d-65a04625e4d4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c81d9d93-6822-4884-836d-65a04625e4d4" (UID: "c81d9d93-6822-4884-836d-65a04625e4d4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:15:20.502098 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.502074    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c81d9d93-6822-4884-836d-65a04625e4d4-kube-api-access-pfk6h" (OuterVolumeSpecName: "kube-api-access-pfk6h") pod "c81d9d93-6822-4884-836d-65a04625e4d4" (UID: "c81d9d93-6822-4884-836d-65a04625e4d4"). InnerVolumeSpecName "kube-api-access-pfk6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:15:20.502098 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.502088    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39c42666-6e6f-4311-b8d3-082f8fae288d-kube-api-access-hrrfx" (OuterVolumeSpecName: "kube-api-access-hrrfx") pod "39c42666-6e6f-4311-b8d3-082f8fae288d" (UID: "39c42666-6e6f-4311-b8d3-082f8fae288d"). InnerVolumeSpecName "kube-api-access-hrrfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:15:20.547388 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.547366    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:20.555421 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.555404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"
Aug 04 09:15:20.601568 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.601531    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfk6h\" (UniqueName: \"kubernetes.io/projected/c81d9d93-6822-4884-836d-65a04625e4d4-kube-api-access-pfk6h\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:15:20.601568 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.601570    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrrfx\" (UniqueName: \"kubernetes.io/projected/39c42666-6e6f-4311-b8d3-082f8fae288d-kube-api-access-hrrfx\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:15:20.601865 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.601585    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81d9d93-6822-4884-836d-65a04625e4d4-extensions-socket-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:15:20.700041 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.699993    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"]
Aug 04 09:15:20.702376 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:15:20.702346    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66505920_6d99_49fe_949b_2e064b56b667.slice/crio-8f5c7b561bec83d1d3db89937ed14d25e94bb292bb38ed22f20d218655429d88 WatchSource:0}: Error finding container 8f5c7b561bec83d1d3db89937ed14d25e94bb292bb38ed22f20d218655429d88: Status 404 returned error can't find the container with id 8f5c7b561bec83d1d3db89937ed14d25e94bb292bb38ed22f20d218655429d88
Aug 04 09:15:20.713711 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:20.713686    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"]
Aug 04 09:15:20.716265 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:15:20.716245    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c7f699_5bf8_4a8f_abc2_9b2b0dfcfaba.slice/crio-ca81b9930ad6c2cb123dab3c0de3eba7fa73e036f377137ea520e3067458f8e5 WatchSource:0}: Error finding container ca81b9930ad6c2cb123dab3c0de3eba7fa73e036f377137ea520e3067458f8e5: Status 404 returned error can't find the container with id ca81b9930ad6c2cb123dab3c0de3eba7fa73e036f377137ea520e3067458f8e5
Aug 04 09:15:21.050404 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.050369    2558 generic.go:356] "Generic (PLEG): container finished" podID="39c42666-6e6f-4311-b8d3-082f8fae288d" containerID="e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2" exitCode=0
Aug 04 09:15:21.050897 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.050416    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:21.050897 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.050468    2558 scope.go:117] "RemoveContainer" containerID="e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2"
Aug 04 09:15:21.052304 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.052276    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv" event={"ID":"98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba","Type":"ContainerStarted","Data":"88aa28761bfb89d9f0e9190d884dfecdd4d878538ae76853aca61fd0e74341a3"}
Aug 04 09:15:21.052423 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.052333    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv" event={"ID":"98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba","Type":"ContainerStarted","Data":"ca81b9930ad6c2cb123dab3c0de3eba7fa73e036f377137ea520e3067458f8e5"}
Aug 04 09:15:21.052423 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.052362    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"
Aug 04 09:15:21.052872 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.052848    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:21.053934 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.053905    2558 generic.go:356] "Generic (PLEG): container finished" podID="c81d9d93-6822-4884-836d-65a04625e4d4" containerID="3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c" exitCode=0
Aug 04 09:15:21.054031 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.053941    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:21.054642 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.054616    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:21.056082 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.055894    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw" event={"ID":"66505920-6d99-49fe-949b-2e064b56b667","Type":"ContainerStarted","Data":"bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425"}
Aug 04 09:15:21.056082 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.055942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw" event={"ID":"66505920-6d99-49fe-949b-2e064b56b667","Type":"ContainerStarted","Data":"8f5c7b561bec83d1d3db89937ed14d25e94bb292bb38ed22f20d218655429d88"}
Aug 04 09:15:21.056082 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.056050    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:21.056430 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.056401    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:21.060055 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.060039    2558 scope.go:117] "RemoveContainer" containerID="e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2"
Aug 04 09:15:21.060330 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.060306    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2\": container with ID starting with e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2 not found: ID does not exist" containerID="e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2"
Aug 04 09:15:21.060411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.060341    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2"} err="failed to get container status \"e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2\": rpc error: code = NotFound desc = could not find container \"e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2\": container with ID starting with e0482857b924bee6a1f5d7e028b232459bbfd8963cbf0c109a98234f9cc0b8e2 not found: ID does not exist"
Aug 04 09:15:21.060411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.060359    2558 scope.go:117] "RemoveContainer" containerID="3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c"
Aug 04 09:15:21.067655 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.067638    2558 scope.go:117] "RemoveContainer" containerID="3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c"
Aug 04 09:15:21.067910 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.067893    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c\": container with ID starting with 3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c not found: ID does not exist" containerID="3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c"
Aug 04 09:15:21.067980 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.067917    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c"} err="failed to get container status \"3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c\": rpc error: code = NotFound desc = could not find container \"3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c\": container with ID starting with 3591e4596884c79401f0621a5120993ebc4d651919f25199e06161651068d32c not found: ID does not exist"
Aug 04 09:15:21.086065 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.086038    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:21.086310 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.086271    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv" podStartSLOduration=1.08625849 podStartE2EDuration="1.08625849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:15:21.084120575 +0000 UTC m=+452.342077647" watchObservedRunningTime="2026-08-04 09:15:21.08625849 +0000 UTC m=+452.344215543"
Aug 04 09:15:21.087776 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.087722    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-jpsvl\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-jpsvl"
Aug 04 09:15:21.114942 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.114898    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw" podStartSLOduration=1.114883089 podStartE2EDuration="1.114883089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:15:21.113587108 +0000 UTC m=+452.371544162" watchObservedRunningTime="2026-08-04 09:15:21.114883089 +0000 UTC m=+452.372840143"
Aug 04 09:15:21.119595 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:21.119570    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-9tj5v\" is forbidden: User \"system:node:ip-10-0-143-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-106.ec2.internal' and this object" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-9tj5v"
Aug 04 09:15:21.294949 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.294923    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39c42666-6e6f-4311-b8d3-082f8fae288d" path="/var/lib/kubelet/pods/39c42666-6e6f-4311-b8d3-082f8fae288d/volumes"
Aug 04 09:15:21.295282 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:21.295268    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c81d9d93-6822-4884-836d-65a04625e4d4" path="/var/lib/kubelet/pods/c81d9d93-6822-4884-836d-65a04625e4d4/volumes"
Aug 04 09:15:32.064515 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:32.064477    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:32.064952 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:32.064552    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zgsbv"
Aug 04 09:15:36.837231 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:36.837200    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"]
Aug 04 09:15:36.837779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:36.837475    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw" podUID="66505920-6d99-49fe-949b-2e064b56b667" containerName="manager" containerID="cri-o://bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425" gracePeriod=10
Aug 04 09:15:37.079686 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.079664    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:37.124058 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.123975    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwv6s\" (UniqueName: \"kubernetes.io/projected/66505920-6d99-49fe-949b-2e064b56b667-kube-api-access-rwv6s\") pod \"66505920-6d99-49fe-949b-2e064b56b667\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") "
Aug 04 09:15:37.124058 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.124039    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66505920-6d99-49fe-949b-2e064b56b667-extensions-socket-volume\") pod \"66505920-6d99-49fe-949b-2e064b56b667\" (UID: \"66505920-6d99-49fe-949b-2e064b56b667\") "
Aug 04 09:15:37.124389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.124366    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66505920-6d99-49fe-949b-2e064b56b667-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "66505920-6d99-49fe-949b-2e064b56b667" (UID: "66505920-6d99-49fe-949b-2e064b56b667"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:15:37.126376 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.126351    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66505920-6d99-49fe-949b-2e064b56b667-kube-api-access-rwv6s" (OuterVolumeSpecName: "kube-api-access-rwv6s") pod "66505920-6d99-49fe-949b-2e064b56b667" (UID: "66505920-6d99-49fe-949b-2e064b56b667"). InnerVolumeSpecName "kube-api-access-rwv6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:15:37.129599 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.129577    2558 generic.go:356] "Generic (PLEG): container finished" podID="66505920-6d99-49fe-949b-2e064b56b667" containerID="bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425" exitCode=0
Aug 04 09:15:37.129700 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.129636    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw" event={"ID":"66505920-6d99-49fe-949b-2e064b56b667","Type":"ContainerDied","Data":"bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425"}
Aug 04 09:15:37.129700 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.129620    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"
Aug 04 09:15:37.129700 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.129682    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw" event={"ID":"66505920-6d99-49fe-949b-2e064b56b667","Type":"ContainerDied","Data":"8f5c7b561bec83d1d3db89937ed14d25e94bb292bb38ed22f20d218655429d88"}
Aug 04 09:15:37.129954 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.129712    2558 scope.go:117] "RemoveContainer" containerID="bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425"
Aug 04 09:15:37.138687 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.138673    2558 scope.go:117] "RemoveContainer" containerID="bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425"
Aug 04 09:15:37.139004 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:15:37.138984    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425\": container with ID starting with bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425 not found: ID does not exist" containerID="bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425"
Aug 04 09:15:37.139069 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.139011    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425"} err="failed to get container status \"bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425\": rpc error: code = NotFound desc = could not find container \"bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425\": container with ID starting with bb568fc4ea764d4c76fba097f8c78ad6dc87a49b6c24555e07d48183d01ac425 not found: ID does not exist"
Aug 04 09:15:37.156389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.156367    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"]
Aug 04 09:15:37.159943 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.159924    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x4ppw"]
Aug 04 09:15:37.224894 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.224868    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwv6s\" (UniqueName: \"kubernetes.io/projected/66505920-6d99-49fe-949b-2e064b56b667-kube-api-access-rwv6s\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:15:37.224894 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.224894    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66505920-6d99-49fe-949b-2e064b56b667-extensions-socket-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:15:37.295189 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:37.295161    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66505920-6d99-49fe-949b-2e064b56b667" path="/var/lib/kubelet/pods/66505920-6d99-49fe-949b-2e064b56b667/volumes"
Aug 04 09:15:53.236580 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.236545    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"]
Aug 04 09:15:53.236947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.236845    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66505920-6d99-49fe-949b-2e064b56b667" containerName="manager"
Aug 04 09:15:53.236947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.236856    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="66505920-6d99-49fe-949b-2e064b56b667" containerName="manager"
Aug 04 09:15:53.237030 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.236951    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="66505920-6d99-49fe-949b-2e064b56b667" containerName="manager"
Aug 04 09:15:53.240949 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.240928    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.243330 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.243311    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-c89dv\""
Aug 04 09:15:53.255936 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.255907    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"]
Aug 04 09:15:53.353505 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353516    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353604    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353927 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353707    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353927 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353816    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8xrr\" (UniqueName: \"kubernetes.io/projected/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-kube-api-access-s8xrr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.353927 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.353876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.454440 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.454409    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.454440 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.454441    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.454647 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.454465    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.454647 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.454483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.454647 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.454507    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xrr\" (UniqueName: \"kubernetes.io/projected/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-kube-api-access-s8xrr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.454647 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.454535    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455183    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455232    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455361    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.455638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.458018    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.462115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.458219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.465719 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.465691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.466336 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.466310    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8xrr\" (UniqueName: \"kubernetes.io/projected/a4e8f32d-06b7-4a1a-b864-4661b8c330d6-kube-api-access-s8xrr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-b96pj\" (UID: \"a4e8f32d-06b7-4a1a-b864-4661b8c330d6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.552944 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.552872    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:53.693432 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:53.693410    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"]
Aug 04 09:15:53.695695 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:15:53.695668    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e8f32d_06b7_4a1a_b864_4661b8c330d6.slice/crio-5ddc8c0ce296af94e9e8428fc8ff2253f55575565c4d692f6a38d981be8d3e36 WatchSource:0}: Error finding container 5ddc8c0ce296af94e9e8428fc8ff2253f55575565c4d692f6a38d981be8d3e36: Status 404 returned error can't find the container with id 5ddc8c0ce296af94e9e8428fc8ff2253f55575565c4d692f6a38d981be8d3e36
Aug 04 09:15:54.206725 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:54.206693    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" event={"ID":"a4e8f32d-06b7-4a1a-b864-4661b8c330d6","Type":"ContainerStarted","Data":"5ddc8c0ce296af94e9e8428fc8ff2253f55575565c4d692f6a38d981be8d3e36"}
Aug 04 09:15:56.341758 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:56.341698    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:15:56.342033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:56.341796    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:15:56.342033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:56.341828    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:15:57.222745 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:57.222696    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" event={"ID":"a4e8f32d-06b7-4a1a-b864-4661b8c330d6","Type":"ContainerStarted","Data":"435fa83e741c75f721608079a698290cbb47c9364238797400ea56be121af5bc"}
Aug 04 09:15:57.243618 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:57.243570    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" podStartSLOduration=1.6004243169999999 podStartE2EDuration="4.243554257s" totalImagesPullingTime="2.64312994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:15:57.242971705 +0000 UTC m=+488.500928761" watchObservedRunningTime="2026-08-04 09:15:57.243554257 +0000 UTC m=+488.501511312"
Aug 04 09:15:57.554073 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:57.554001    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:57.558517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:57.558489    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:58.194085 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.194046    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"]
Aug 04 09:15:58.197493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.197472    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.199983 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.199965    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fvztm\""
Aug 04 09:15:58.200095 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.199969    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 04 09:15:58.206714 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.206687    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"]
Aug 04 09:15:58.227283 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.227258    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:58.228314 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.228295    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:15:58.291360 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.291325    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"]
Aug 04 09:15:58.297114 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.297087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnjmx\" (UniqueName: \"kubernetes.io/projected/a7baf9e8-7bac-4857-a94b-4979a0608a67-kube-api-access-gnjmx\") pod \"limitador-limitador-859c4d96d6-t2n2h\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.297266 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.297166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7baf9e8-7bac-4857-a94b-4979a0608a67-config-file\") pod \"limitador-limitador-859c4d96d6-t2n2h\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.398166 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.398136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnjmx\" (UniqueName: \"kubernetes.io/projected/a7baf9e8-7bac-4857-a94b-4979a0608a67-kube-api-access-gnjmx\") pod \"limitador-limitador-859c4d96d6-t2n2h\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.398351 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.398181    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7baf9e8-7bac-4857-a94b-4979a0608a67-config-file\") pod \"limitador-limitador-859c4d96d6-t2n2h\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.398797 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.398767    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7baf9e8-7bac-4857-a94b-4979a0608a67-config-file\") pod \"limitador-limitador-859c4d96d6-t2n2h\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.406279 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.406237    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnjmx\" (UniqueName: \"kubernetes.io/projected/a7baf9e8-7bac-4857-a94b-4979a0608a67-kube-api-access-gnjmx\") pod \"limitador-limitador-859c4d96d6-t2n2h\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.510355 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.510284    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:15:58.651279 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.651241    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:15:58.655681 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.655655    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:58.662676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.662613    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"]
Aug 04 09:15:58.666253 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.664809    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:15:58.689602 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.689576    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:15:58.700495 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.700471    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldb9r\" (UniqueName: \"kubernetes.io/projected/22cd4245-7cef-4ff7-be23-42d6099d2a61-kube-api-access-ldb9r\") pod \"limitador-limitador-85d9db4d5d-ksvrz\" (UID: \"22cd4245-7cef-4ff7-be23-42d6099d2a61\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:58.700582 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.700500    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/22cd4245-7cef-4ff7-be23-42d6099d2a61-config-file\") pod \"limitador-limitador-85d9db4d5d-ksvrz\" (UID: \"22cd4245-7cef-4ff7-be23-42d6099d2a61\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:58.801258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.801196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldb9r\" (UniqueName: \"kubernetes.io/projected/22cd4245-7cef-4ff7-be23-42d6099d2a61-kube-api-access-ldb9r\") pod \"limitador-limitador-85d9db4d5d-ksvrz\" (UID: \"22cd4245-7cef-4ff7-be23-42d6099d2a61\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:58.801258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.801230    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/22cd4245-7cef-4ff7-be23-42d6099d2a61-config-file\") pod \"limitador-limitador-85d9db4d5d-ksvrz\" (UID: \"22cd4245-7cef-4ff7-be23-42d6099d2a61\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:58.801843 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.801826    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/22cd4245-7cef-4ff7-be23-42d6099d2a61-config-file\") pod \"limitador-limitador-85d9db4d5d-ksvrz\" (UID: \"22cd4245-7cef-4ff7-be23-42d6099d2a61\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:58.809479 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.809458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldb9r\" (UniqueName: \"kubernetes.io/projected/22cd4245-7cef-4ff7-be23-42d6099d2a61-kube-api-access-ldb9r\") pod \"limitador-limitador-85d9db4d5d-ksvrz\" (UID: \"22cd4245-7cef-4ff7-be23-42d6099d2a61\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:58.980058 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:58.980031    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:15:59.113069 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:59.113045    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:15:59.115558 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:15:59.115528    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22cd4245_7cef_4ff7_be23_42d6099d2a61.slice/crio-2fb1e51e999a05b9d7babe2bfa651b63d69ba84aea166b6ac62f498c68e4d6ec WatchSource:0}: Error finding container 2fb1e51e999a05b9d7babe2bfa651b63d69ba84aea166b6ac62f498c68e4d6ec: Status 404 returned error can't find the container with id 2fb1e51e999a05b9d7babe2bfa651b63d69ba84aea166b6ac62f498c68e4d6ec
Aug 04 09:15:59.232509 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:59.232477    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h" event={"ID":"a7baf9e8-7bac-4857-a94b-4979a0608a67","Type":"ContainerStarted","Data":"5b0aef355ae206c9e335593aa35ce151d4ee35a46096a1134fa59426f4112281"}
Aug 04 09:15:59.233584 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:15:59.233564    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz" event={"ID":"22cd4245-7cef-4ff7-be23-42d6099d2a61","Type":"ContainerStarted","Data":"2fb1e51e999a05b9d7babe2bfa651b63d69ba84aea166b6ac62f498c68e4d6ec"}
Aug 04 09:16:00.239702 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:00.239617    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h" event={"ID":"a7baf9e8-7bac-4857-a94b-4979a0608a67","Type":"ContainerStarted","Data":"04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064"}
Aug 04 09:16:00.240136 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:00.239931    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:16:00.258191 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:00.258140    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h" podStartSLOduration=1.00581284 podStartE2EDuration="2.258126316s" totalImagesPullingTime="1.252313476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:16:00.256712126 +0000 UTC m=+491.514669184" watchObservedRunningTime="2026-08-04 09:16:00.258126316 +0000 UTC m=+491.516083370"
Aug 04 09:16:01.246214 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:01.246175    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz" event={"ID":"22cd4245-7cef-4ff7-be23-42d6099d2a61","Type":"ContainerStarted","Data":"6630ec9428aeed7818932a2f89338b5414ae0b8ae41ef747781fbcc6459cb8af"}
Aug 04 09:16:01.264344 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:01.264293    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz" podStartSLOduration=1.329818932 podStartE2EDuration="3.264278197s" totalImagesPullingTime="1.934459265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:16:01.263102564 +0000 UTC m=+492.521059619" watchObservedRunningTime="2026-08-04 09:16:01.264278197 +0000 UTC m=+492.522235248"
Aug 04 09:16:02.250344 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:02.250317    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:16:11.247233 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:11.247197    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:16:13.255514 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:13.255487    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"
Aug 04 09:16:13.298809 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:13.298771    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"]
Aug 04 09:16:13.298991 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:13.298965    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h" podUID="a7baf9e8-7bac-4857-a94b-4979a0608a67" containerName="limitador" containerID="cri-o://04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064" gracePeriod=30
Aug 04 09:16:14.238088 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.238063    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:16:14.302342 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.302309    2558 generic.go:356] "Generic (PLEG): container finished" podID="a7baf9e8-7bac-4857-a94b-4979a0608a67" containerID="04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064" exitCode=0
Aug 04 09:16:14.302779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.302361    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"
Aug 04 09:16:14.302779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.302408    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h" event={"ID":"a7baf9e8-7bac-4857-a94b-4979a0608a67","Type":"ContainerDied","Data":"04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064"}
Aug 04 09:16:14.302779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.302479    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-t2n2h" event={"ID":"a7baf9e8-7bac-4857-a94b-4979a0608a67","Type":"ContainerDied","Data":"5b0aef355ae206c9e335593aa35ce151d4ee35a46096a1134fa59426f4112281"}
Aug 04 09:16:14.302779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.302512    2558 scope.go:117] "RemoveContainer" containerID="04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064"
Aug 04 09:16:14.310950 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.310913    2558 scope.go:117] "RemoveContainer" containerID="04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064"
Aug 04 09:16:14.311160 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:16:14.311140    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064\": container with ID starting with 04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064 not found: ID does not exist" containerID="04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064"
Aug 04 09:16:14.311231 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.311185    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064"} err="failed to get container status \"04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064\": rpc error: code = NotFound desc = could not find container \"04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064\": container with ID starting with 04017138a23804e55c1bc413a39c73d935fa8944ed0716559338542ac9041064 not found: ID does not exist"
Aug 04 09:16:14.325097 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.324940    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnjmx\" (UniqueName: \"kubernetes.io/projected/a7baf9e8-7bac-4857-a94b-4979a0608a67-kube-api-access-gnjmx\") pod \"a7baf9e8-7bac-4857-a94b-4979a0608a67\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") "
Aug 04 09:16:14.325097 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.325073    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7baf9e8-7bac-4857-a94b-4979a0608a67-config-file\") pod \"a7baf9e8-7bac-4857-a94b-4979a0608a67\" (UID: \"a7baf9e8-7bac-4857-a94b-4979a0608a67\") "
Aug 04 09:16:14.325586 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.325560    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7baf9e8-7bac-4857-a94b-4979a0608a67-config-file" (OuterVolumeSpecName: "config-file") pod "a7baf9e8-7bac-4857-a94b-4979a0608a67" (UID: "a7baf9e8-7bac-4857-a94b-4979a0608a67"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:16:14.332268 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.332227    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7baf9e8-7bac-4857-a94b-4979a0608a67-kube-api-access-gnjmx" (OuterVolumeSpecName: "kube-api-access-gnjmx") pod "a7baf9e8-7bac-4857-a94b-4979a0608a67" (UID: "a7baf9e8-7bac-4857-a94b-4979a0608a67"). InnerVolumeSpecName "kube-api-access-gnjmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:16:14.425820 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.425757    2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7baf9e8-7bac-4857-a94b-4979a0608a67-config-file\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:16:14.425820 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.425781    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnjmx\" (UniqueName: \"kubernetes.io/projected/a7baf9e8-7bac-4857-a94b-4979a0608a67-kube-api-access-gnjmx\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:16:14.622838 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.622809    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"]
Aug 04 09:16:14.627749 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.627712    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-t2n2h"]
Aug 04 09:16:14.741353 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.741286    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"]
Aug 04 09:16:14.741627 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.741614    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7baf9e8-7bac-4857-a94b-4979a0608a67" containerName="limitador"
Aug 04 09:16:14.741667 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.741629    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7baf9e8-7bac-4857-a94b-4979a0608a67" containerName="limitador"
Aug 04 09:16:14.741704 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.741693    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7baf9e8-7bac-4857-a94b-4979a0608a67" containerName="limitador"
Aug 04 09:16:14.745977 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.745959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:14.750158 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.750137    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 09:16:14.750288 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.750137    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 04 09:16:14.750288 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.750193    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zsx9k\""
Aug 04 09:16:14.750288 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.750207    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Aug 04 09:16:14.758972 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.758952    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"]
Aug 04 09:16:14.829185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.829157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9bmx\" (UniqueName: \"kubernetes.io/projected/a2f1a578-5c90-4546-a28a-2a73660378ea-kube-api-access-z9bmx\") pod \"postgres-69fb4594cf-l9dq8\" (UID: \"a2f1a578-5c90-4546-a28a-2a73660378ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:14.829338 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.829210    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a2f1a578-5c90-4546-a28a-2a73660378ea-data\") pod \"postgres-69fb4594cf-l9dq8\" (UID: \"a2f1a578-5c90-4546-a28a-2a73660378ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:14.930538 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.930499    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a2f1a578-5c90-4546-a28a-2a73660378ea-data\") pod \"postgres-69fb4594cf-l9dq8\" (UID: \"a2f1a578-5c90-4546-a28a-2a73660378ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:14.930687 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.930597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9bmx\" (UniqueName: \"kubernetes.io/projected/a2f1a578-5c90-4546-a28a-2a73660378ea-kube-api-access-z9bmx\") pod \"postgres-69fb4594cf-l9dq8\" (UID: \"a2f1a578-5c90-4546-a28a-2a73660378ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:14.930925 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.930904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a2f1a578-5c90-4546-a28a-2a73660378ea-data\") pod \"postgres-69fb4594cf-l9dq8\" (UID: \"a2f1a578-5c90-4546-a28a-2a73660378ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:14.939554 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:14.939531    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9bmx\" (UniqueName: \"kubernetes.io/projected/a2f1a578-5c90-4546-a28a-2a73660378ea-kube-api-access-z9bmx\") pod \"postgres-69fb4594cf-l9dq8\" (UID: \"a2f1a578-5c90-4546-a28a-2a73660378ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:15.056371 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:15.056304    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:15.189151 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:15.189119    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"]
Aug 04 09:16:15.192615 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:16:15.192585    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f1a578_5c90_4546_a28a_2a73660378ea.slice/crio-73674adc40b45c66c096ae1820612b4b09b06fedd09be3e6c6a163935cd6d713 WatchSource:0}: Error finding container 73674adc40b45c66c096ae1820612b4b09b06fedd09be3e6c6a163935cd6d713: Status 404 returned error can't find the container with id 73674adc40b45c66c096ae1820612b4b09b06fedd09be3e6c6a163935cd6d713
Aug 04 09:16:15.295402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:15.295365    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7baf9e8-7bac-4857-a94b-4979a0608a67" path="/var/lib/kubelet/pods/a7baf9e8-7bac-4857-a94b-4979a0608a67/volumes"
Aug 04 09:16:15.307951 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:15.307873    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8" event={"ID":"a2f1a578-5c90-4546-a28a-2a73660378ea","Type":"ContainerStarted","Data":"73674adc40b45c66c096ae1820612b4b09b06fedd09be3e6c6a163935cd6d713"}
Aug 04 09:16:20.341810 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:20.341765    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8" event={"ID":"a2f1a578-5c90-4546-a28a-2a73660378ea","Type":"ContainerStarted","Data":"7cae9a067fb0a7318968e5954236743e0111af7e0567a9a65678daa6861d75ca"}
Aug 04 09:16:20.342242 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:20.341887    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:20.356656 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:20.356602    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8" podStartSLOduration=1.422041089 podStartE2EDuration="6.356584162s" totalImagesPullingTime="4.934543073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:16:20.355875968 +0000 UTC m=+511.613833069" watchObservedRunningTime="2026-08-04 09:16:20.356584162 +0000 UTC m=+511.614541217"
Aug 04 09:16:26.376775 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:26.376715    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l9dq8"
Aug 04 09:16:33.490399 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.490359    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-748fb55845-p8g6w"]
Aug 04 09:16:33.519652 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.519613    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-748fb55845-p8g6w"]
Aug 04 09:16:33.519827 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.519787    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.523322 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.523296    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lf45h\""
Aug 04 09:16:33.523477 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.523330    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 09:16:33.523477 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.523296    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 09:16:33.694182 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.694144    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95b4\" (UniqueName: \"kubernetes.io/projected/9f665a74-3991-40de-943c-6fd204b812cf-kube-api-access-l95b4\") pod \"maas-controller-748fb55845-p8g6w\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") " pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.694345 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.694239    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f665a74-3991-40de-943c-6fd204b812cf-webhook-cert\") pod \"maas-controller-748fb55845-p8g6w\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") " pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.795482 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.795413    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f665a74-3991-40de-943c-6fd204b812cf-webhook-cert\") pod \"maas-controller-748fb55845-p8g6w\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") " pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.795482 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.795470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95b4\" (UniqueName: \"kubernetes.io/projected/9f665a74-3991-40de-943c-6fd204b812cf-kube-api-access-l95b4\") pod \"maas-controller-748fb55845-p8g6w\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") " pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.797889 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.797870    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f665a74-3991-40de-943c-6fd204b812cf-webhook-cert\") pod \"maas-controller-748fb55845-p8g6w\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") " pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.804171 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.804151    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95b4\" (UniqueName: \"kubernetes.io/projected/9f665a74-3991-40de-943c-6fd204b812cf-kube-api-access-l95b4\") pod \"maas-controller-748fb55845-p8g6w\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") " pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.830099 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.830068    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:33.956959 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:33.956929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-748fb55845-p8g6w"]
Aug 04 09:16:33.960357 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:16:33.960336    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f665a74_3991_40de_943c_6fd204b812cf.slice/crio-e4d6c3dc4c8e415ddf18095b5d58a695e89fb5037516f7cb9782ff725df4eb36 WatchSource:0}: Error finding container e4d6c3dc4c8e415ddf18095b5d58a695e89fb5037516f7cb9782ff725df4eb36: Status 404 returned error can't find the container with id e4d6c3dc4c8e415ddf18095b5d58a695e89fb5037516f7cb9782ff725df4eb36
Aug 04 09:16:34.398220 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:34.398179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748fb55845-p8g6w" event={"ID":"9f665a74-3991-40de-943c-6fd204b812cf","Type":"ContainerStarted","Data":"e4d6c3dc4c8e415ddf18095b5d58a695e89fb5037516f7cb9782ff725df4eb36"}
Aug 04 09:16:36.409854 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:36.409817    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748fb55845-p8g6w" event={"ID":"9f665a74-3991-40de-943c-6fd204b812cf","Type":"ContainerStarted","Data":"eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51"}
Aug 04 09:16:36.410220 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:36.409956    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:36.437103 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:36.437052    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-748fb55845-p8g6w" podStartSLOduration=1.093412021 podStartE2EDuration="3.43703501s" totalImagesPullingTime="2.343622989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:16:36.435061483 +0000 UTC m=+527.693018533" watchObservedRunningTime="2026-08-04 09:16:36.43703501 +0000 UTC m=+527.694992064"
Aug 04 09:16:47.416287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:47.416208    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:48.160091 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.160054    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-748fb55845-p8g6w"]
Aug 04 09:16:48.160301 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.160272    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-748fb55845-p8g6w" podUID="9f665a74-3991-40de-943c-6fd204b812cf" containerName="manager" containerID="cri-o://eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51" gracePeriod=30
Aug 04 09:16:48.406590 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.406567    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:48.462108 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.462032    2558 generic.go:356] "Generic (PLEG): container finished" podID="9f665a74-3991-40de-943c-6fd204b812cf" containerID="eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51" exitCode=0
Aug 04 09:16:48.462108 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.462074    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-748fb55845-p8g6w"
Aug 04 09:16:48.462574 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.462119    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748fb55845-p8g6w" event={"ID":"9f665a74-3991-40de-943c-6fd204b812cf","Type":"ContainerDied","Data":"eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51"}
Aug 04 09:16:48.462574 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.462163    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-748fb55845-p8g6w" event={"ID":"9f665a74-3991-40de-943c-6fd204b812cf","Type":"ContainerDied","Data":"e4d6c3dc4c8e415ddf18095b5d58a695e89fb5037516f7cb9782ff725df4eb36"}
Aug 04 09:16:48.462574 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.462185    2558 scope.go:117] "RemoveContainer" containerID="eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51"
Aug 04 09:16:48.471089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.471071    2558 scope.go:117] "RemoveContainer" containerID="eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51"
Aug 04 09:16:48.471379 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:16:48.471359    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51\": container with ID starting with eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51 not found: ID does not exist" containerID="eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51"
Aug 04 09:16:48.471457 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.471392    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51"} err="failed to get container status \"eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51\": rpc error: code = NotFound desc = could not find container \"eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51\": container with ID starting with eb4c1374dd251e3f8227bd0e1608e28337967c684df0f06b9906569ec2e98e51 not found: ID does not exist"
Aug 04 09:16:48.518948 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.518923    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l95b4\" (UniqueName: \"kubernetes.io/projected/9f665a74-3991-40de-943c-6fd204b812cf-kube-api-access-l95b4\") pod \"9f665a74-3991-40de-943c-6fd204b812cf\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") "
Aug 04 09:16:48.519055 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.518961    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f665a74-3991-40de-943c-6fd204b812cf-webhook-cert\") pod \"9f665a74-3991-40de-943c-6fd204b812cf\" (UID: \"9f665a74-3991-40de-943c-6fd204b812cf\") "
Aug 04 09:16:48.521052 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.521028    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f665a74-3991-40de-943c-6fd204b812cf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9f665a74-3991-40de-943c-6fd204b812cf" (UID: "9f665a74-3991-40de-943c-6fd204b812cf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:16:48.521147 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.521112    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f665a74-3991-40de-943c-6fd204b812cf-kube-api-access-l95b4" (OuterVolumeSpecName: "kube-api-access-l95b4") pod "9f665a74-3991-40de-943c-6fd204b812cf" (UID: "9f665a74-3991-40de-943c-6fd204b812cf"). InnerVolumeSpecName "kube-api-access-l95b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:16:48.620323 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.620293    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l95b4\" (UniqueName: \"kubernetes.io/projected/9f665a74-3991-40de-943c-6fd204b812cf-kube-api-access-l95b4\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:16:48.620323 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.620317    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f665a74-3991-40de-943c-6fd204b812cf-webhook-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:16:48.784526 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.784494    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-748fb55845-p8g6w"]
Aug 04 09:16:48.788955 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:48.788932    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-748fb55845-p8g6w"]
Aug 04 09:16:49.295234 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:16:49.295209    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f665a74-3991-40de-943c-6fd204b812cf" path="/var/lib/kubelet/pods/9f665a74-3991-40de-943c-6fd204b812cf/volumes"
Aug 04 09:17:12.734120 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.734086    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-kbzjm"]
Aug 04 09:17:12.734568 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.734432    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f665a74-3991-40de-943c-6fd204b812cf" containerName="manager"
Aug 04 09:17:12.734568 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.734443    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f665a74-3991-40de-943c-6fd204b812cf" containerName="manager"
Aug 04 09:17:12.734568 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.734493    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f665a74-3991-40de-943c-6fd204b812cf" containerName="manager"
Aug 04 09:17:12.736334 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.736314    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:12.740126 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.740101    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bxljn\""
Aug 04 09:17:12.740265 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.740138    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 04 09:17:12.752573 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.752551    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-kbzjm"]
Aug 04 09:17:12.807049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.807013    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21cb922-e090-4055-b1a7-371acf923f6d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-kbzjm\" (UID: \"d21cb922-e090-4055-b1a7-371acf923f6d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:12.807199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.807060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlvqp\" (UniqueName: \"kubernetes.io/projected/d21cb922-e090-4055-b1a7-371acf923f6d-kube-api-access-dlvqp\") pod \"payload-processing-5cc6cd444b-kbzjm\" (UID: \"d21cb922-e090-4055-b1a7-371acf923f6d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:12.812916 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.812891    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:17:12.908281 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.908250    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21cb922-e090-4055-b1a7-371acf923f6d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-kbzjm\" (UID: \"d21cb922-e090-4055-b1a7-371acf923f6d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:12.908467 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.908293    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlvqp\" (UniqueName: \"kubernetes.io/projected/d21cb922-e090-4055-b1a7-371acf923f6d-kube-api-access-dlvqp\") pod \"payload-processing-5cc6cd444b-kbzjm\" (UID: \"d21cb922-e090-4055-b1a7-371acf923f6d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:12.909063 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.909031    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d21cb922-e090-4055-b1a7-371acf923f6d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-kbzjm\" (UID: \"d21cb922-e090-4055-b1a7-371acf923f6d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:12.917466 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:12.917442    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlvqp\" (UniqueName: \"kubernetes.io/projected/d21cb922-e090-4055-b1a7-371acf923f6d-kube-api-access-dlvqp\") pod \"payload-processing-5cc6cd444b-kbzjm\" (UID: \"d21cb922-e090-4055-b1a7-371acf923f6d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:13.046167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:13.046083    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:13.181697 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:13.181634    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-kbzjm"]
Aug 04 09:17:13.184192 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:17:13.184167    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21cb922_e090_4055_b1a7_371acf923f6d.slice/crio-47359515f38978ed647ef567bafd3fa0e48561009cfb6428dda478d4732fae7f WatchSource:0}: Error finding container 47359515f38978ed647ef567bafd3fa0e48561009cfb6428dda478d4732fae7f: Status 404 returned error can't find the container with id 47359515f38978ed647ef567bafd3fa0e48561009cfb6428dda478d4732fae7f
Aug 04 09:17:13.571006 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:13.570967    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm" event={"ID":"d21cb922-e090-4055-b1a7-371acf923f6d","Type":"ContainerStarted","Data":"47359515f38978ed647ef567bafd3fa0e48561009cfb6428dda478d4732fae7f"}
Aug 04 09:17:16.586557 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:16.586514    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm" event={"ID":"d21cb922-e090-4055-b1a7-371acf923f6d","Type":"ContainerStarted","Data":"ca063fa5528d6b5e7826598bd51c537fbc1538875fe2655b35500033565376b6"}
Aug 04 09:17:16.586979 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:16.586671    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:16.603119 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:16.603071    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm" podStartSLOduration=1.712689745 podStartE2EDuration="4.603057034s" totalImagesPullingTime="2.890367289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:17:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:17:16.601600609 +0000 UTC m=+567.859557674" watchObservedRunningTime="2026-08-04 09:17:16.603057034 +0000 UTC m=+567.861014066"
Aug 04 09:17:22.614996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:22.614964    2558 generic.go:356] "Generic (PLEG): container finished" podID="a4e8f32d-06b7-4a1a-b864-4661b8c330d6" containerID="435fa83e741c75f721608079a698290cbb47c9364238797400ea56be121af5bc" exitCode=0
Aug 04 09:17:22.614996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:22.615000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" event={"ID":"a4e8f32d-06b7-4a1a-b864-4661b8c330d6","Type":"ContainerDied","Data":"435fa83e741c75f721608079a698290cbb47c9364238797400ea56be121af5bc"}
Aug 04 09:17:22.615499 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:22.615335    2558 scope.go:117] "RemoveContainer" containerID="435fa83e741c75f721608079a698290cbb47c9364238797400ea56be121af5bc"
Aug 04 09:17:22.616286 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:22.616254    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:17:22.616351 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:22.616336    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:17:22.616389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:22.616373    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:17:23.622135 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:23.622102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" event={"ID":"a4e8f32d-06b7-4a1a-b864-4661b8c330d6","Type":"ContainerStarted","Data":"d9b59aece228ff44ac8558d8130a48dae4122e403b860dfcca1ebc874091491d"}
Aug 04 09:17:24.553451 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:24.553405    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:17:24.554282 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:24.554253    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b96pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Aug 04 09:17:24.554402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:24.554312    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" podUID="a4e8f32d-06b7-4a1a-b864-4661b8c330d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Aug 04 09:17:25.553296 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:25.553251    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b96pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Aug 04 09:17:25.553808 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:25.553350    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" podUID="a4e8f32d-06b7-4a1a-b864-4661b8c330d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Aug 04 09:17:26.553236 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:26.553201    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b96pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Aug 04 09:17:26.553410 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:26.553260    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" podUID="a4e8f32d-06b7-4a1a-b864-4661b8c330d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Aug 04 09:17:27.553359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:27.553317    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b96pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Aug 04 09:17:27.553559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:27.553378    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" podUID="a4e8f32d-06b7-4a1a-b864-4661b8c330d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Aug 04 09:17:27.592709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:27.592684    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-kbzjm"
Aug 04 09:17:28.228123 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:28.228060    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:17:28.557040 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:28.556940    2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-b96pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Aug 04 09:17:28.557040 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:28.557003    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj" podUID="a4e8f32d-06b7-4a1a-b864-4661b8c330d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Aug 04 09:17:29.557101 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:29.557074    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:17:29.558004 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:29.557990    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-b96pj"
Aug 04 09:17:36.762375 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:36.762342    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:17:49.731454 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.731422    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"]
Aug 04 09:17:49.739800 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.739779    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.743364 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.743342    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 04 09:17:49.743364 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.743355    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 04 09:17:49.743513 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.743357    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pfrgq\""
Aug 04 09:17:49.743513 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.743349    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Aug 04 09:17:49.749263 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.749242    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"]
Aug 04 09:17:49.834991 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.834961    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.835141 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.835010    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.835141 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.835086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg79s\" (UniqueName: \"kubernetes.io/projected/d9d34197-686b-417e-8685-f39fc6cc5115-kube-api-access-kg79s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.835141 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.835123    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.835241 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.835147    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.835241 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.835166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d34197-686b-417e-8685-f39fc6cc5115-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.935846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.935809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg79s\" (UniqueName: \"kubernetes.io/projected/d9d34197-686b-417e-8685-f39fc6cc5115-kube-api-access-kg79s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.935846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.935852    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.936048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.935878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.936048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.935997    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d34197-686b-417e-8685-f39fc6cc5115-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.936122 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.936107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.936186 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.936174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.936285 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.936266    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.936469 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.936449    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.936527 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.936499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.938229 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.938209    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d34197-686b-417e-8685-f39fc6cc5115-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.938565 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.938547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d34197-686b-417e-8685-f39fc6cc5115-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:49.943769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:49.943728    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg79s\" (UniqueName: \"kubernetes.io/projected/d9d34197-686b-417e-8685-f39fc6cc5115-kube-api-access-kg79s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nznrm\" (UID: \"d9d34197-686b-417e-8685-f39fc6cc5115\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:50.051479 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:50.051413    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:17:50.130166 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:50.130135    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:17:50.186247 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:50.186224    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"]
Aug 04 09:17:50.188789 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:17:50.188759    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d34197_686b_417e_8685_f39fc6cc5115.slice/crio-ee1372a8126ddca2669cdadbb5e4bf2410875e20a980a10d7a0d1c085fa7856b WatchSource:0}: Error finding container ee1372a8126ddca2669cdadbb5e4bf2410875e20a980a10d7a0d1c085fa7856b: Status 404 returned error can't find the container with id ee1372a8126ddca2669cdadbb5e4bf2410875e20a980a10d7a0d1c085fa7856b
Aug 04 09:17:50.754779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:50.754714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm" event={"ID":"d9d34197-686b-417e-8685-f39fc6cc5115","Type":"ContainerStarted","Data":"ee1372a8126ddca2669cdadbb5e4bf2410875e20a980a10d7a0d1c085fa7856b"}
Aug 04 09:17:51.761392 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:51.761353    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm" event={"ID":"d9d34197-686b-417e-8685-f39fc6cc5115","Type":"ContainerStarted","Data":"ff7ff6b6a9f82662b0203b1c6f2f721afa926431b9dbbdf5a2a1703fc49c43f0"}
Aug 04 09:17:51.780274 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:17:51.780212    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm" podStartSLOduration=1.6819664859999999 podStartE2EDuration="2.780194994s" totalImagesPullingTime="1.098228508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:17:51.778412836 +0000 UTC m=+603.036369884" watchObservedRunningTime="2026-08-04 09:17:51.780194994 +0000 UTC m=+603.038152049"
Aug 04 09:18:00.052431 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:00.052386    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:18:00.052832 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:00.052457    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:18:00.064914 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:00.064888    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:18:00.241240 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:00.241210    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:18:00.811947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:00.811921    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nznrm"
Aug 04 09:18:10.534553 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:10.534463    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:18:26.547812 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:26.547781    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:18:29.531983 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.531950    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"]
Aug 04 09:18:29.536424 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.536407    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.540195 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.540166    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Aug 04 09:18:29.551521 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.551496    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"]
Aug 04 09:18:29.571444 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.571418    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca2b29b1-689d-4d42-8a11-c9d9b1717728-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.571564 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.571463    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.571636 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.571581    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrnj9\" (UniqueName: \"kubernetes.io/projected/ca2b29b1-689d-4d42-8a11-c9d9b1717728-kube-api-access-jrnj9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.571636 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.571621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.571759 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.571717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.571815 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.571782    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672410 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrnj9\" (UniqueName: \"kubernetes.io/projected/ca2b29b1-689d-4d42-8a11-c9d9b1717728-kube-api-access-jrnj9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672410 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672441    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672461    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672486    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca2b29b1-689d-4d42-8a11-c9d9b1717728-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672833 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672819    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.672943 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672921    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.673009 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.672953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.675035 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.675012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca2b29b1-689d-4d42-8a11-c9d9b1717728-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.675261 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.675245    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca2b29b1-689d-4d42-8a11-c9d9b1717728-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.682289 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.682268    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrnj9\" (UniqueName: \"kubernetes.io/projected/ca2b29b1-689d-4d42-8a11-c9d9b1717728-kube-api-access-jrnj9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4\" (UID: \"ca2b29b1-689d-4d42-8a11-c9d9b1717728\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.853345 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.853312    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:29.983124 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.983102    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"]
Aug 04 09:18:29.985148 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:18:29.985117    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca2b29b1_689d_4d42_8a11_c9d9b1717728.slice/crio-dc79ea365973c702762c7b704f421d4e599721ffdecc3972d15a45fed8e5f9ef WatchSource:0}: Error finding container dc79ea365973c702762c7b704f421d4e599721ffdecc3972d15a45fed8e5f9ef: Status 404 returned error can't find the container with id dc79ea365973c702762c7b704f421d4e599721ffdecc3972d15a45fed8e5f9ef
Aug 04 09:18:29.986508 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:29.986488    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 09:18:30.934331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:30.934300    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4" event={"ID":"ca2b29b1-689d-4d42-8a11-c9d9b1717728","Type":"ContainerStarted","Data":"964314191f3e352a08ef081ff9e60cc4598af703cbed0aed2a53e46fee68fc8c"}
Aug 04 09:18:30.934331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:30.934339    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4" event={"ID":"ca2b29b1-689d-4d42-8a11-c9d9b1717728","Type":"ContainerStarted","Data":"dc79ea365973c702762c7b704f421d4e599721ffdecc3972d15a45fed8e5f9ef"}
Aug 04 09:18:30.953606 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:30.953563    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4" podStartSLOduration=1.763412089 podStartE2EDuration="1.953549274s" totalImagesPullingTime="190.137185ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:18:30.952029796 +0000 UTC m=+642.209986851" watchObservedRunningTime="2026-08-04 09:18:30.953549274 +0000 UTC m=+642.211506327"
Aug 04 09:18:37.033920 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:37.033886    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:18:39.853932 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:39.853894    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:39.854418 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:39.854193    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:39.866708 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:39.866685    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:18:39.985001 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:18:39.984975    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4"
Aug 04 09:19:56.932922 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:19:56.932838    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:20:05.736842 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:20:05.736798    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:20:18.459975 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:20:18.459944    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:21:16.972175 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:16.972085    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-fb65c789d-b77qn"]
Aug 04 09:21:16.975427 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:16.975405    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:16.978138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:16.978118    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 09:21:16.979130 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:16.979108    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 09:21:16.979210 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:16.979155    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lf45h\""
Aug 04 09:21:16.986802 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:16.986780    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-fb65c789d-b77qn"]
Aug 04 09:21:17.084908 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.084880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q2jm\" (UniqueName: \"kubernetes.io/projected/1d24672a-171c-4fc1-a644-5831cfedb32b-kube-api-access-7q2jm\") pod \"maas-controller-fb65c789d-b77qn\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") " pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:17.085049 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.084999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d24672a-171c-4fc1-a644-5831cfedb32b-webhook-cert\") pod \"maas-controller-fb65c789d-b77qn\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") " pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:17.186216 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.186191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d24672a-171c-4fc1-a644-5831cfedb32b-webhook-cert\") pod \"maas-controller-fb65c789d-b77qn\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") " pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:17.186356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.186234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q2jm\" (UniqueName: \"kubernetes.io/projected/1d24672a-171c-4fc1-a644-5831cfedb32b-kube-api-access-7q2jm\") pod \"maas-controller-fb65c789d-b77qn\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") " pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:17.188919 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.188896    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d24672a-171c-4fc1-a644-5831cfedb32b-webhook-cert\") pod \"maas-controller-fb65c789d-b77qn\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") " pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:17.195270 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.195245    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q2jm\" (UniqueName: \"kubernetes.io/projected/1d24672a-171c-4fc1-a644-5831cfedb32b-kube-api-access-7q2jm\") pod \"maas-controller-fb65c789d-b77qn\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") " pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:17.285786 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.285681    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:17.426466 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.426441    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-fb65c789d-b77qn"]
Aug 04 09:21:17.429238 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:21:17.429206    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d24672a_171c_4fc1_a644_5831cfedb32b.slice/crio-f109b73b8f7716c9a55f923630aaec7579093f9d367be04f3f15e8d188483803 WatchSource:0}: Error finding container f109b73b8f7716c9a55f923630aaec7579093f9d367be04f3f15e8d188483803: Status 404 returned error can't find the container with id f109b73b8f7716c9a55f923630aaec7579093f9d367be04f3f15e8d188483803
Aug 04 09:21:17.654698 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:17.654668    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fb65c789d-b77qn" event={"ID":"1d24672a-171c-4fc1-a644-5831cfedb32b","Type":"ContainerStarted","Data":"f109b73b8f7716c9a55f923630aaec7579093f9d367be04f3f15e8d188483803"}
Aug 04 09:21:18.660242 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:18.660210    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fb65c789d-b77qn" event={"ID":"1d24672a-171c-4fc1-a644-5831cfedb32b","Type":"ContainerStarted","Data":"35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b"}
Aug 04 09:21:18.660601 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:18.660330    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:18.678497 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:18.678451    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-fb65c789d-b77qn" podStartSLOduration=2.229678468 podStartE2EDuration="2.678438428s" totalImagesPullingTime="448.75996ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:21:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:21:18.677693724 +0000 UTC m=+809.935650783" watchObservedRunningTime="2026-08-04 09:21:18.678438428 +0000 UTC m=+809.936395481"
Aug 04 09:21:29.667313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:29.667283    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:34.280544 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.280506    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-fb65c789d-b77qn"]
Aug 04 09:21:34.280961 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.280702    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-fb65c789d-b77qn" podUID="1d24672a-171c-4fc1-a644-5831cfedb32b" containerName="manager" containerID="cri-o://35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b" gracePeriod=30
Aug 04 09:21:34.525691 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.525663    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:34.636071 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.636039    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q2jm\" (UniqueName: \"kubernetes.io/projected/1d24672a-171c-4fc1-a644-5831cfedb32b-kube-api-access-7q2jm\") pod \"1d24672a-171c-4fc1-a644-5831cfedb32b\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") "
Aug 04 09:21:34.636248 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.636079    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d24672a-171c-4fc1-a644-5831cfedb32b-webhook-cert\") pod \"1d24672a-171c-4fc1-a644-5831cfedb32b\" (UID: \"1d24672a-171c-4fc1-a644-5831cfedb32b\") "
Aug 04 09:21:34.638188 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.638160    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d24672a-171c-4fc1-a644-5831cfedb32b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1d24672a-171c-4fc1-a644-5831cfedb32b" (UID: "1d24672a-171c-4fc1-a644-5831cfedb32b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:21:34.638357 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.638329    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d24672a-171c-4fc1-a644-5831cfedb32b-kube-api-access-7q2jm" (OuterVolumeSpecName: "kube-api-access-7q2jm") pod "1d24672a-171c-4fc1-a644-5831cfedb32b" (UID: "1d24672a-171c-4fc1-a644-5831cfedb32b"). InnerVolumeSpecName "kube-api-access-7q2jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:21:34.732428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.732395    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d24672a-171c-4fc1-a644-5831cfedb32b" containerID="35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b" exitCode=0
Aug 04 09:21:34.732557 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.732440    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fb65c789d-b77qn"
Aug 04 09:21:34.732557 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.732480    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fb65c789d-b77qn" event={"ID":"1d24672a-171c-4fc1-a644-5831cfedb32b","Type":"ContainerDied","Data":"35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b"}
Aug 04 09:21:34.732557 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.732534    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fb65c789d-b77qn" event={"ID":"1d24672a-171c-4fc1-a644-5831cfedb32b","Type":"ContainerDied","Data":"f109b73b8f7716c9a55f923630aaec7579093f9d367be04f3f15e8d188483803"}
Aug 04 09:21:34.732656 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.732559    2558 scope.go:117] "RemoveContainer" containerID="35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b"
Aug 04 09:21:34.737267 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.737248    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q2jm\" (UniqueName: \"kubernetes.io/projected/1d24672a-171c-4fc1-a644-5831cfedb32b-kube-api-access-7q2jm\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:21:34.737267 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.737268    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d24672a-171c-4fc1-a644-5831cfedb32b-webhook-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:21:34.741512 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.741498    2558 scope.go:117] "RemoveContainer" containerID="35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b"
Aug 04 09:21:34.741782 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:21:34.741761    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b\": container with ID starting with 35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b not found: ID does not exist" containerID="35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b"
Aug 04 09:21:34.741832 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.741793    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b"} err="failed to get container status \"35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b\": rpc error: code = NotFound desc = could not find container \"35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b\": container with ID starting with 35c9421caf8820651af684ca16d4ae279b33f85f72380e69830f6f190fb0af3b not found: ID does not exist"
Aug 04 09:21:34.753483 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.753460    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-fb65c789d-b77qn"]
Aug 04 09:21:34.759524 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:34.759501    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-fb65c789d-b77qn"]
Aug 04 09:21:35.295157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.295126    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d24672a-171c-4fc1-a644-5831cfedb32b" path="/var/lib/kubelet/pods/1d24672a-171c-4fc1-a644-5831cfedb32b/volumes"
Aug 04 09:21:35.557959 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.557890    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-fb65c789d-rvt85"]
Aug 04 09:21:35.558248 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.558235    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d24672a-171c-4fc1-a644-5831cfedb32b" containerName="manager"
Aug 04 09:21:35.558287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.558250    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d24672a-171c-4fc1-a644-5831cfedb32b" containerName="manager"
Aug 04 09:21:35.558322 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.558305    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d24672a-171c-4fc1-a644-5831cfedb32b" containerName="manager"
Aug 04 09:21:35.562672 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.562650    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:35.564997 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.564982    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 09:21:35.564997 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.564989    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 09:21:35.566051 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.566033    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lf45h\""
Aug 04 09:21:35.570459 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.570439    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-fb65c789d-rvt85"]
Aug 04 09:21:35.651400 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.651370    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dq9s\" (UniqueName: \"kubernetes.io/projected/f61f28fc-b699-4998-8869-f42dbdc6ce88-kube-api-access-2dq9s\") pod \"maas-controller-fb65c789d-rvt85\" (UID: \"f61f28fc-b699-4998-8869-f42dbdc6ce88\") " pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:35.651530 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.651410    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f61f28fc-b699-4998-8869-f42dbdc6ce88-webhook-cert\") pod \"maas-controller-fb65c789d-rvt85\" (UID: \"f61f28fc-b699-4998-8869-f42dbdc6ce88\") " pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:35.752361 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.752330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dq9s\" (UniqueName: \"kubernetes.io/projected/f61f28fc-b699-4998-8869-f42dbdc6ce88-kube-api-access-2dq9s\") pod \"maas-controller-fb65c789d-rvt85\" (UID: \"f61f28fc-b699-4998-8869-f42dbdc6ce88\") " pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:35.752514 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.752382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f61f28fc-b699-4998-8869-f42dbdc6ce88-webhook-cert\") pod \"maas-controller-fb65c789d-rvt85\" (UID: \"f61f28fc-b699-4998-8869-f42dbdc6ce88\") " pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:35.754974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.754947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f61f28fc-b699-4998-8869-f42dbdc6ce88-webhook-cert\") pod \"maas-controller-fb65c789d-rvt85\" (UID: \"f61f28fc-b699-4998-8869-f42dbdc6ce88\") " pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:35.760444 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.760422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dq9s\" (UniqueName: \"kubernetes.io/projected/f61f28fc-b699-4998-8869-f42dbdc6ce88-kube-api-access-2dq9s\") pod \"maas-controller-fb65c789d-rvt85\" (UID: \"f61f28fc-b699-4998-8869-f42dbdc6ce88\") " pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:35.874089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:35.874058    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:36.004298 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:36.004267    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-fb65c789d-rvt85"]
Aug 04 09:21:36.007506 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:21:36.007463    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61f28fc_b699_4998_8869_f42dbdc6ce88.slice/crio-ecfa0dcedfd281b5e2098ca296b76fd5dc5696ac9237b41ea8f4c032ca3bc2cd WatchSource:0}: Error finding container ecfa0dcedfd281b5e2098ca296b76fd5dc5696ac9237b41ea8f4c032ca3bc2cd: Status 404 returned error can't find the container with id ecfa0dcedfd281b5e2098ca296b76fd5dc5696ac9237b41ea8f4c032ca3bc2cd
Aug 04 09:21:36.743666 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:36.743590    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fb65c789d-rvt85" event={"ID":"f61f28fc-b699-4998-8869-f42dbdc6ce88","Type":"ContainerStarted","Data":"2bde8372748dfcd2d1ef4bd9c92488cfe3fff6a8d846c9fca25607ef327a5ee0"}
Aug 04 09:21:36.743666 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:36.743636    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fb65c789d-rvt85" event={"ID":"f61f28fc-b699-4998-8869-f42dbdc6ce88","Type":"ContainerStarted","Data":"ecfa0dcedfd281b5e2098ca296b76fd5dc5696ac9237b41ea8f4c032ca3bc2cd"}
Aug 04 09:21:36.743666 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:36.743665    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:36.763347 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:36.763300    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-fb65c789d-rvt85" podStartSLOduration=1.4171339029999999 podStartE2EDuration="1.763284157s" totalImagesPullingTime="346.150254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:21:36.761879576 +0000 UTC m=+828.019836629" watchObservedRunningTime="2026-08-04 09:21:36.763284157 +0000 UTC m=+828.021241211"
Aug 04 09:21:47.750645 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:47.750618    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-fb65c789d-rvt85"
Aug 04 09:21:57.946428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:21:57.946394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:22:07.145205 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:22:07.145171    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:23:06.534711 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:23:06.534624    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:23:13.532387 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:23:13.532353    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:23:29.235498 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:23:29.235468    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:23:34.841415 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:23:34.841377    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:23:44.540059 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:23:44.540014    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:23:50.945299 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:23:50.945263    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:24:05.134452 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:24:05.134417    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:24:10.035098 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:24:10.035066    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:24:15.028380 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:24:15.028349    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:24:19.839966 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:24:19.839937    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:24:27.628341 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:24:27.628306    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:24:40.231478 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:24:40.231443    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:24:50.828001 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:24:50.827960    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:25:14.832572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:25:14.832537    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:25:25.138624 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:25:25.138588    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:25:31.034187 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:25:31.034151    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:25:38.437715 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:25:38.437680    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:25:47.044438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:25:47.044350    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:25:52.130031 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:25:52.130001    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:25:57.228193 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:25:57.228157    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:26:03.233869 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:26:03.233816    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:26:15.340623 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:26:15.340589    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:26:24.327071 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:26:24.327038    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:26:32.143511 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:26:32.143476    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:26:44.939826 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:26:44.939791    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:26:53.934839 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:26:53.934802    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:27:00.539248 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:27:00.539214    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:27:09.934928 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:27:09.934892    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:29:24.826621 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:29:24.826589    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:29:29.931259 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:29:29.931222    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:29:55.440254 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:29:55.440218    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:30:15.528324 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:30:15.528244    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:30:21.739831 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:30:21.739790    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:30:30.436331 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:30:30.436300    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:30:35.646125 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:30:35.646076    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:30:46.030309 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:30:46.030251    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:30:51.150511 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:30:51.150478    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:31:01.243467 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:31:01.243431    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:31:07.136199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:31:07.136161    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:31:16.532820 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:31:16.532785    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:31:23.134559 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:31:23.134522    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:31:32.642541 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:31:32.642505    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:31:57.036239 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:31:57.036145    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:32:02.436667 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:02.436630    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:32:15.259233 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.259196    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"]
Aug 04 09:32:15.262761 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.262724    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.265234 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.265212    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-417cca6a-kserve-self-signed-certs\""
Aug 04 09:32:15.278260 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.278234    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"]
Aug 04 09:32:15.395846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.395811    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-home\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.396007 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.395854    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-tmp-dir\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.396007 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.395872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b50c0532-d308-41a1-8934-82e95602c6be-tls-certs\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.396100 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.396014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-model-cache\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.396100 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.396043    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-dshm\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.396100 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.396065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr6kn\" (UniqueName: \"kubernetes.io/projected/b50c0532-d308-41a1-8934-82e95602c6be-kube-api-access-pr6kn\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497125 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497091    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-tmp-dir\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497125 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b50c0532-d308-41a1-8934-82e95602c6be-tls-certs\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497367 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-model-cache\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497367 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-dshm\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497367 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497236    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr6kn\" (UniqueName: \"kubernetes.io/projected/b50c0532-d308-41a1-8934-82e95602c6be-kube-api-access-pr6kn\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497367 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-home\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497592 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497569    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-tmp-dir\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497653 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497611    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-model-cache\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.497716 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.497699    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-home\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.499615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.499588    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-dshm\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.500026 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.500006    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b50c0532-d308-41a1-8934-82e95602c6be-tls-certs\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.505054 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.505036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr6kn\" (UniqueName: \"kubernetes.io/projected/b50c0532-d308-41a1-8934-82e95602c6be-kube-api-access-pr6kn\") pod \"e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") " pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.571303 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.571236    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:15.701647 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:15.701611    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"]
Aug 04 09:32:15.704462 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:32:15.704435    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb50c0532_d308_41a1_8934_82e95602c6be.slice/crio-952d4447bdc4dcd848840e1968ae9e31a63c42f05ab9d66f50a803f46902dc96 WatchSource:0}: Error finding container 952d4447bdc4dcd848840e1968ae9e31a63c42f05ab9d66f50a803f46902dc96: Status 404 returned error can't find the container with id 952d4447bdc4dcd848840e1968ae9e31a63c42f05ab9d66f50a803f46902dc96
Aug 04 09:32:16.624484 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:16.624430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8" event={"ID":"b50c0532-d308-41a1-8934-82e95602c6be","Type":"ContainerStarted","Data":"ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610"}
Aug 04 09:32:16.624484 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:16.624488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8" event={"ID":"b50c0532-d308-41a1-8934-82e95602c6be","Type":"ContainerStarted","Data":"952d4447bdc4dcd848840e1968ae9e31a63c42f05ab9d66f50a803f46902dc96"}
Aug 04 09:32:16.644833 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:16.644790    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8" podStartSLOduration=1.6447752279999999 podStartE2EDuration="1.644775228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:32:16.643046963 +0000 UTC m=+1467.901004017" watchObservedRunningTime="2026-08-04 09:32:16.644775228 +0000 UTC m=+1467.902732292"
Aug 04 09:32:17.446378 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:17.446337    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:32:25.572121 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:25.572036    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:25.572121 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:25.572075    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:25.584832 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:25.584805    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:25.678458 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:25.678432    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:32:34.340405 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:34.340372    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:32:48.873878 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:48.873827    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"]
Aug 04 09:32:51.839077 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:32:51.839042    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:01.753337 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:01.753300    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:03.893857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:03.893814    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8" podUID="b50c0532-d308-41a1-8934-82e95602c6be" containerName="main" containerID="cri-o://ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610" gracePeriod=45
Aug 04 09:33:04.149258 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.149200    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:33:04.209494 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209464    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr6kn\" (UniqueName: \"kubernetes.io/projected/b50c0532-d308-41a1-8934-82e95602c6be-kube-api-access-pr6kn\") pod \"b50c0532-d308-41a1-8934-82e95602c6be\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") "
Aug 04 09:33:04.209616 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209548    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-dshm\") pod \"b50c0532-d308-41a1-8934-82e95602c6be\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") "
Aug 04 09:33:04.209616 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209593    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-tmp-dir\") pod \"b50c0532-d308-41a1-8934-82e95602c6be\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") "
Aug 04 09:33:04.209616 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209615    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-home\") pod \"b50c0532-d308-41a1-8934-82e95602c6be\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") "
Aug 04 09:33:04.209802 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209640    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-model-cache\") pod \"b50c0532-d308-41a1-8934-82e95602c6be\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") "
Aug 04 09:33:04.209802 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209680    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b50c0532-d308-41a1-8934-82e95602c6be-tls-certs\") pod \"b50c0532-d308-41a1-8934-82e95602c6be\" (UID: \"b50c0532-d308-41a1-8934-82e95602c6be\") "
Aug 04 09:33:04.209933 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209898    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b50c0532-d308-41a1-8934-82e95602c6be" (UID: "b50c0532-d308-41a1-8934-82e95602c6be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:33:04.209933 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209883    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-home" (OuterVolumeSpecName: "home") pod "b50c0532-d308-41a1-8934-82e95602c6be" (UID: "b50c0532-d308-41a1-8934-82e95602c6be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:33:04.210057 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.209952    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-model-cache" (OuterVolumeSpecName: "model-cache") pod "b50c0532-d308-41a1-8934-82e95602c6be" (UID: "b50c0532-d308-41a1-8934-82e95602c6be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:33:04.211857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.211836    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b50c0532-d308-41a1-8934-82e95602c6be-kube-api-access-pr6kn" (OuterVolumeSpecName: "kube-api-access-pr6kn") pod "b50c0532-d308-41a1-8934-82e95602c6be" (UID: "b50c0532-d308-41a1-8934-82e95602c6be"). InnerVolumeSpecName "kube-api-access-pr6kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:33:04.211927 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.211895    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-dshm" (OuterVolumeSpecName: "dshm") pod "b50c0532-d308-41a1-8934-82e95602c6be" (UID: "b50c0532-d308-41a1-8934-82e95602c6be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:33:04.212006 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.211983    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b50c0532-d308-41a1-8934-82e95602c6be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b50c0532-d308-41a1-8934-82e95602c6be" (UID: "b50c0532-d308-41a1-8934-82e95602c6be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:33:04.311160 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.311125    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-tmp-dir\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:33:04.311160 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.311153    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-home\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:33:04.311318 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.311167    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-model-cache\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:33:04.311318 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.311178    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b50c0532-d308-41a1-8934-82e95602c6be-tls-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:33:04.311318 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.311189    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr6kn\" (UniqueName: \"kubernetes.io/projected/b50c0532-d308-41a1-8934-82e95602c6be-kube-api-access-pr6kn\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:33:04.311318 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.311200    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b50c0532-d308-41a1-8934-82e95602c6be-dshm\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:33:04.849897 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.849862    2558 generic.go:356] "Generic (PLEG): container finished" podID="b50c0532-d308-41a1-8934-82e95602c6be" containerID="ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610" exitCode=0
Aug 04 09:33:04.850076 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.849912    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8" event={"ID":"b50c0532-d308-41a1-8934-82e95602c6be","Type":"ContainerDied","Data":"ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610"}
Aug 04 09:33:04.850076 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.849921    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"
Aug 04 09:33:04.850076 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.849944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8" event={"ID":"b50c0532-d308-41a1-8934-82e95602c6be","Type":"ContainerDied","Data":"952d4447bdc4dcd848840e1968ae9e31a63c42f05ab9d66f50a803f46902dc96"}
Aug 04 09:33:04.850076 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.849961    2558 scope.go:117] "RemoveContainer" containerID="ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610"
Aug 04 09:33:04.862289 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.862267    2558 scope.go:117] "RemoveContainer" containerID="ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610"
Aug 04 09:33:04.862567 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:33:04.862548    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610\": container with ID starting with ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610 not found: ID does not exist" containerID="ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610"
Aug 04 09:33:04.862617 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.862577    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610"} err="failed to get container status \"ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610\": rpc error: code = NotFound desc = could not find container \"ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610\": container with ID starting with ef8b9498814aca6290eccd9df74054fca578a2ea8f5b3bac642f0017d1b4b610 not found: ID does not exist"
Aug 04 09:33:04.872070 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.872045    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"]
Aug 04 09:33:04.875994 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:04.875970    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-417cca6a-kserve-6857664967-8d6f8"]
Aug 04 09:33:05.295040 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:05.294967    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b50c0532-d308-41a1-8934-82e95602c6be" path="/var/lib/kubelet/pods/b50c0532-d308-41a1-8934-82e95602c6be/volumes"
Aug 04 09:33:07.341934 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:07.341907    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:17.241262 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:17.241226    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:22.533912 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:22.533878    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:29.739415 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:29.739377    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:39.348001 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:39.347967    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:46.434777 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:46.434723    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:33:58.941949 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:33:58.941912    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:34:04.230079 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:34:04.230048    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:34:22.039501 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:34:22.039466    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:34:27.229814 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:34:27.229782    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:34:40.337273 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:34:40.337192    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:34:45.335776 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:34:45.335724    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:34:52.439130 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:34:52.439084    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:34:57.328376 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:34:57.328339    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:35:03.631693 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:35:03.631650    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:35:13.743092 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:35:13.743061    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:35:23.533814 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:35:23.533782    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:35:29.544044 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:35:29.544007    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:35:38.744634 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:35:38.744597    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:35:45.871543 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:35:45.871508    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:36:01.237056 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:01.237021    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:36:07.243897 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:07.243853    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:36:13.439031 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:13.438944    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:36:25.440480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:25.440446    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:36:33.839109 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:33.839052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:36:39.436397 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:39.436362    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:36:52.686192 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.686160    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"]
Aug 04 09:36:52.686588 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.686506    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b50c0532-d308-41a1-8934-82e95602c6be" containerName="main"
Aug 04 09:36:52.686588 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.686518    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b50c0532-d308-41a1-8934-82e95602c6be" containerName="main"
Aug 04 09:36:52.686588 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.686571    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b50c0532-d308-41a1-8934-82e95602c6be" containerName="main"
Aug 04 09:36:52.689396 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.689380    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:52.693356 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.693335    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e842246f\""
Aug 04 09:36:52.693637 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.693622    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e842246f-dockercfg-c6twr\""
Aug 04 09:36:52.700499 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.700476    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"]
Aug 04 09:36:52.835773 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.835713    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rht5q\" (UniqueName: \"kubernetes.io/projected/0dfd8697-6cef-4476-8c3c-176764018e3d-kube-api-access-rht5q\") pod \"payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:52.835959 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.835793    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dfd8697-6cef-4476-8c3c-176764018e3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:52.936566 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.936475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rht5q\" (UniqueName: \"kubernetes.io/projected/0dfd8697-6cef-4476-8c3c-176764018e3d-kube-api-access-rht5q\") pod \"payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:52.936566 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.936533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dfd8697-6cef-4476-8c3c-176764018e3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:52.937285 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.937261    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dfd8697-6cef-4476-8c3c-176764018e3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:52.946000 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:52.945978    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rht5q\" (UniqueName: \"kubernetes.io/projected/0dfd8697-6cef-4476-8c3c-176764018e3d-kube-api-access-rht5q\") pod \"payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:53.000683 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:53.000655    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:53.134441 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:53.134368    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"]
Aug 04 09:36:53.136697 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:36:53.136673    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dfd8697_6cef_4476_8c3c_176764018e3d.slice/crio-c61b3bb0de64a8ed7aa98de3facbc013d62fde8a56c879fe9a8db794157c5f60 WatchSource:0}: Error finding container c61b3bb0de64a8ed7aa98de3facbc013d62fde8a56c879fe9a8db794157c5f60: Status 404 returned error can't find the container with id c61b3bb0de64a8ed7aa98de3facbc013d62fde8a56c879fe9a8db794157c5f60
Aug 04 09:36:53.895000 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:53.894959    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc" event={"ID":"0dfd8697-6cef-4476-8c3c-176764018e3d","Type":"ContainerStarted","Data":"4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5"}
Aug 04 09:36:53.895368 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:53.895012    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc" event={"ID":"0dfd8697-6cef-4476-8c3c-176764018e3d","Type":"ContainerStarted","Data":"c61b3bb0de64a8ed7aa98de3facbc013d62fde8a56c879fe9a8db794157c5f60"}
Aug 04 09:36:53.895368 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:53.895055    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:36:53.913930 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:53.913872    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc" podStartSLOduration=1.913858005 podStartE2EDuration="1.913858005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:36:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:36:53.911793027 +0000 UTC m=+1745.169750082" watchObservedRunningTime="2026-08-04 09:36:53.913858005 +0000 UTC m=+1745.171815058"
Aug 04 09:36:55.344903 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:36:55.344876    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:37:04.901820 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:04.901784    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:37:32.572840 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:32.572802    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"]
Aug 04 09:37:32.573411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:32.573242    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc" podUID="0dfd8697-6cef-4476-8c3c-176764018e3d" containerName="payload-pre-processing" containerID="cri-o://4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5" gracePeriod=30
Aug 04 09:37:32.830154 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:32.830135    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:37:32.965516 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:32.965475    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dfd8697-6cef-4476-8c3c-176764018e3d-plugins-config-volume\") pod \"0dfd8697-6cef-4476-8c3c-176764018e3d\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") "
Aug 04 09:37:32.965682 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:32.965538    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rht5q\" (UniqueName: \"kubernetes.io/projected/0dfd8697-6cef-4476-8c3c-176764018e3d-kube-api-access-rht5q\") pod \"0dfd8697-6cef-4476-8c3c-176764018e3d\" (UID: \"0dfd8697-6cef-4476-8c3c-176764018e3d\") "
Aug 04 09:37:32.965931 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:32.965907    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dfd8697-6cef-4476-8c3c-176764018e3d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0dfd8697-6cef-4476-8c3c-176764018e3d" (UID: "0dfd8697-6cef-4476-8c3c-176764018e3d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:37:32.968179 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:32.968112    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dfd8697-6cef-4476-8c3c-176764018e3d-kube-api-access-rht5q" (OuterVolumeSpecName: "kube-api-access-rht5q") pod "0dfd8697-6cef-4476-8c3c-176764018e3d" (UID: "0dfd8697-6cef-4476-8c3c-176764018e3d"). InnerVolumeSpecName "kube-api-access-rht5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:37:33.067110 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.067066    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dfd8697-6cef-4476-8c3c-176764018e3d-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:37:33.067110 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.067107    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rht5q\" (UniqueName: \"kubernetes.io/projected/0dfd8697-6cef-4476-8c3c-176764018e3d-kube-api-access-rht5q\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:37:33.075965 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.075938    2558 generic.go:356] "Generic (PLEG): container finished" podID="0dfd8697-6cef-4476-8c3c-176764018e3d" containerID="4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5" exitCode=0
Aug 04 09:37:33.076086 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.075983    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"
Aug 04 09:37:33.076086 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.076024    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc" event={"ID":"0dfd8697-6cef-4476-8c3c-176764018e3d","Type":"ContainerDied","Data":"4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5"}
Aug 04 09:37:33.076156 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.076086    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc" event={"ID":"0dfd8697-6cef-4476-8c3c-176764018e3d","Type":"ContainerDied","Data":"c61b3bb0de64a8ed7aa98de3facbc013d62fde8a56c879fe9a8db794157c5f60"}
Aug 04 09:37:33.076156 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.076115    2558 scope.go:117] "RemoveContainer" containerID="4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5"
Aug 04 09:37:33.084934 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.084913    2558 scope.go:117] "RemoveContainer" containerID="4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5"
Aug 04 09:37:33.085202 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:37:33.085183    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5\": container with ID starting with 4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5 not found: ID does not exist" containerID="4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5"
Aug 04 09:37:33.085326 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.085216    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5"} err="failed to get container status \"4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5\": rpc error: code = NotFound desc = could not find container \"4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5\": container with ID starting with 4d2546d4b529f152d19f26bbf273246aafb8071bc552a90fdfb888b61ef2b3b5 not found: ID does not exist"
Aug 04 09:37:33.097297 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.097272    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"]
Aug 04 09:37:33.102037 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.102017    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e842246f-844bc88464-jhpzc"]
Aug 04 09:37:33.295215 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:33.295183    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dfd8697-6cef-4476-8c3c-176764018e3d" path="/var/lib/kubelet/pods/0dfd8697-6cef-4476-8c3c-176764018e3d/volumes"
Aug 04 09:37:35.537893 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:35.537859    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:37:38.047688 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.047654    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"]
Aug 04 09:37:38.048111 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.048041    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dfd8697-6cef-4476-8c3c-176764018e3d" containerName="payload-pre-processing"
Aug 04 09:37:38.048111 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.048053    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dfd8697-6cef-4476-8c3c-176764018e3d" containerName="payload-pre-processing"
Aug 04 09:37:38.048195 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.048112    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dfd8697-6cef-4476-8c3c-176764018e3d" containerName="payload-pre-processing"
Aug 04 09:37:38.052654 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.052637    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.055303 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.055282    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-93164817\""
Aug 04 09:37:38.055433 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.055285    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-93164817-dockercfg-jmpm9\""
Aug 04 09:37:38.066294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.066271    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"]
Aug 04 09:37:38.100864 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.100839    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"]
Aug 04 09:37:38.104176 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.104160    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.110569 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.110540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c570a6-c4dc-4224-a258-f87df16acec2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") " pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.110657 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.110636    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl6bk\" (UniqueName: \"kubernetes.io/projected/e2c570a6-c4dc-4224-a258-f87df16acec2-kube-api-access-vl6bk\") pod \"payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") " pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.114864 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.114843    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"]
Aug 04 09:37:38.211901 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.211868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c69c2b-0159-407f-bc94-178411e1657b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.212078 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.211972    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c570a6-c4dc-4224-a258-f87df16acec2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") " pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.212078 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.212035    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl6bk\" (UniqueName: \"kubernetes.io/projected/e2c570a6-c4dc-4224-a258-f87df16acec2-kube-api-access-vl6bk\") pod \"payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") " pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.212182 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.212077    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fllr\" (UniqueName: \"kubernetes.io/projected/98c69c2b-0159-407f-bc94-178411e1657b-kube-api-access-4fllr\") pod \"payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.212967 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.212946    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c570a6-c4dc-4224-a258-f87df16acec2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") " pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.220915 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.220889    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl6bk\" (UniqueName: \"kubernetes.io/projected/e2c570a6-c4dc-4224-a258-f87df16acec2-kube-api-access-vl6bk\") pod \"payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") " pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.313505 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.313420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fllr\" (UniqueName: \"kubernetes.io/projected/98c69c2b-0159-407f-bc94-178411e1657b-kube-api-access-4fllr\") pod \"payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.313505 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.313470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c69c2b-0159-407f-bc94-178411e1657b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.314178 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.314158    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c69c2b-0159-407f-bc94-178411e1657b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.321411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.321390    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fllr\" (UniqueName: \"kubernetes.io/projected/98c69c2b-0159-407f-bc94-178411e1657b-kube-api-access-4fllr\") pod \"payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.362925 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.362898    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:38.414341 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.414303    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:38.517762 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.517700    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"]
Aug 04 09:37:38.521750 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:37:38.521704    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c570a6_c4dc_4224_a258_f87df16acec2.slice/crio-005a3fc6d1daa2a1b5a549d34b15bc07d0baf71036e67035e386a259b3ceac90 WatchSource:0}: Error finding container 005a3fc6d1daa2a1b5a549d34b15bc07d0baf71036e67035e386a259b3ceac90: Status 404 returned error can't find the container with id 005a3fc6d1daa2a1b5a549d34b15bc07d0baf71036e67035e386a259b3ceac90
Aug 04 09:37:38.570192 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:38.570168    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"]
Aug 04 09:37:38.572568 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:37:38.572538    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c69c2b_0159_407f_bc94_178411e1657b.slice/crio-f706bad0b4a6540eafca19efce9be557703856a646b712181af1aade712297bb WatchSource:0}: Error finding container f706bad0b4a6540eafca19efce9be557703856a646b712181af1aade712297bb: Status 404 returned error can't find the container with id f706bad0b4a6540eafca19efce9be557703856a646b712181af1aade712297bb
Aug 04 09:37:39.108660 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.108623    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g" event={"ID":"98c69c2b-0159-407f-bc94-178411e1657b","Type":"ContainerStarted","Data":"e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36"}
Aug 04 09:37:39.109087 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.108682    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g" event={"ID":"98c69c2b-0159-407f-bc94-178411e1657b","Type":"ContainerStarted","Data":"f706bad0b4a6540eafca19efce9be557703856a646b712181af1aade712297bb"}
Aug 04 09:37:39.109087 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.108709    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:39.110280 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.110256    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg" event={"ID":"e2c570a6-c4dc-4224-a258-f87df16acec2","Type":"ContainerStarted","Data":"7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86"}
Aug 04 09:37:39.110388 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.110301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg" event={"ID":"e2c570a6-c4dc-4224-a258-f87df16acec2","Type":"ContainerStarted","Data":"005a3fc6d1daa2a1b5a549d34b15bc07d0baf71036e67035e386a259b3ceac90"}
Aug 04 09:37:39.110388 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.110326    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:37:39.127485 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.127433    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g" podStartSLOduration=1.127419954 podStartE2EDuration="1.127419954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:37:39.125474497 +0000 UTC m=+1790.383431551" watchObservedRunningTime="2026-08-04 09:37:39.127419954 +0000 UTC m=+1790.385377008"
Aug 04 09:37:39.144064 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:39.144019    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg" podStartSLOduration=1.144004413 podStartE2EDuration="1.144004413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:37:39.142169488 +0000 UTC m=+1790.400126554" watchObservedRunningTime="2026-08-04 09:37:39.144004413 +0000 UTC m=+1790.401961466"
Aug 04 09:37:43.533438 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:43.533340    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:37:50.116881 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:50.116851    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:37:50.117244 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:37:50.117048    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:38:02.399521 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.399481    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"]
Aug 04 09:38:02.403030 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.403006    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"
Aug 04 09:38:02.405532 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.405513    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\""
Aug 04 09:38:02.411822 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.411799    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"]
Aug 04 09:38:02.429620 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.429594    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9f7247-fecb-49f2-bce8-57bfb0141b8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-93164817-txppr\" (UID: \"5d9f7247-fecb-49f2-bce8-57bfb0141b8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"
Aug 04 09:38:02.531078 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.531045    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9f7247-fecb-49f2-bce8-57bfb0141b8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-93164817-txppr\" (UID: \"5d9f7247-fecb-49f2-bce8-57bfb0141b8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"
Aug 04 09:38:02.531640 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.531622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9f7247-fecb-49f2-bce8-57bfb0141b8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-93164817-txppr\" (UID: \"5d9f7247-fecb-49f2-bce8-57bfb0141b8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"
Aug 04 09:38:02.714974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.714901    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"
Aug 04 09:38:02.852292 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.852245    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"]
Aug 04 09:38:02.854478 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:38:02.854449    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9f7247_fecb_49f2_bce8_57bfb0141b8c.slice/crio-d0aba190a14af78495681e5e7f5f2ab3e432d205bdddc3295598fa249519d248 WatchSource:0}: Error finding container d0aba190a14af78495681e5e7f5f2ab3e432d205bdddc3295598fa249519d248: Status 404 returned error can't find the container with id d0aba190a14af78495681e5e7f5f2ab3e432d205bdddc3295598fa249519d248
Aug 04 09:38:02.856636 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:02.856615    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 09:38:03.232848 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:03.232806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr" event={"ID":"5d9f7247-fecb-49f2-bce8-57bfb0141b8c","Type":"ContainerStarted","Data":"d0aba190a14af78495681e5e7f5f2ab3e432d205bdddc3295598fa249519d248"}
Aug 04 09:38:06.250493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:06.250459    2558 generic.go:356] "Generic (PLEG): container finished" podID="5d9f7247-fecb-49f2-bce8-57bfb0141b8c" containerID="8febc0b6594409151e342e679d531731562c17c739a7b8439c4e9b3b7076a122" exitCode=0
Aug 04 09:38:06.250895 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:06.250509    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr" event={"ID":"5d9f7247-fecb-49f2-bce8-57bfb0141b8c","Type":"ContainerDied","Data":"8febc0b6594409151e342e679d531731562c17c739a7b8439c4e9b3b7076a122"}
Aug 04 09:38:07.382747 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:07.382708    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"
Aug 04 09:38:07.475777 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:07.475721    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9f7247-fecb-49f2-bce8-57bfb0141b8c-maas-api-service-ca\") pod \"5d9f7247-fecb-49f2-bce8-57bfb0141b8c\" (UID: \"5d9f7247-fecb-49f2-bce8-57bfb0141b8c\") "
Aug 04 09:38:07.476056 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:07.476034    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d9f7247-fecb-49f2-bce8-57bfb0141b8c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d9f7247-fecb-49f2-bce8-57bfb0141b8c" (UID: "5d9f7247-fecb-49f2-bce8-57bfb0141b8c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:38:07.576981 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:07.576909    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9f7247-fecb-49f2-bce8-57bfb0141b8c-maas-api-service-ca\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:38:08.260691 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:08.260660    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"
Aug 04 09:38:08.260691 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:08.260669    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr" event={"ID":"5d9f7247-fecb-49f2-bce8-57bfb0141b8c","Type":"ContainerDied","Data":"d0aba190a14af78495681e5e7f5f2ab3e432d205bdddc3295598fa249519d248"}
Aug 04 09:38:08.260968 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:08.260715    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0aba190a14af78495681e5e7f5f2ab3e432d205bdddc3295598fa249519d248"
Aug 04 09:38:18.003237 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.003196    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"]
Aug 04 09:38:18.005642 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.003504    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg" podUID="e2c570a6-c4dc-4224-a258-f87df16acec2" containerName="payload-processing" containerID="cri-o://7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86" gracePeriod=30
Aug 04 09:38:18.010443 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.010416    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"]
Aug 04 09:38:18.011124 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.010668    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g" podUID="98c69c2b-0159-407f-bc94-178411e1657b" containerName="payload-pre-processing" containerID="cri-o://e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36" gracePeriod=30
Aug 04 09:38:18.157111 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.157029    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"]
Aug 04 09:38:18.164324 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.164287    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-93164817-txppr"]
Aug 04 09:38:18.295703 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.295640    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:38:18.300022 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.299989    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:38:18.308102 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.308074    2558 generic.go:356] "Generic (PLEG): container finished" podID="98c69c2b-0159-407f-bc94-178411e1657b" containerID="e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36" exitCode=0
Aug 04 09:38:18.308271 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.308123    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"
Aug 04 09:38:18.308271 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.308163    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g" event={"ID":"98c69c2b-0159-407f-bc94-178411e1657b","Type":"ContainerDied","Data":"e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36"}
Aug 04 09:38:18.308271 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.308205    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g" event={"ID":"98c69c2b-0159-407f-bc94-178411e1657b","Type":"ContainerDied","Data":"f706bad0b4a6540eafca19efce9be557703856a646b712181af1aade712297bb"}
Aug 04 09:38:18.308271 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.308234    2558 scope.go:117] "RemoveContainer" containerID="e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36"
Aug 04 09:38:18.310306 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.310259    2558 generic.go:356] "Generic (PLEG): container finished" podID="e2c570a6-c4dc-4224-a258-f87df16acec2" containerID="7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86" exitCode=0
Aug 04 09:38:18.310402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.310303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg" event={"ID":"e2c570a6-c4dc-4224-a258-f87df16acec2","Type":"ContainerDied","Data":"7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86"}
Aug 04 09:38:18.310402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.310364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg" event={"ID":"e2c570a6-c4dc-4224-a258-f87df16acec2","Type":"ContainerDied","Data":"005a3fc6d1daa2a1b5a549d34b15bc07d0baf71036e67035e386a259b3ceac90"}
Aug 04 09:38:18.310402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.310313    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"
Aug 04 09:38:18.318287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.318261    2558 scope.go:117] "RemoveContainer" containerID="e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36"
Aug 04 09:38:18.318638 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:38:18.318609    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36\": container with ID starting with e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36 not found: ID does not exist" containerID="e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36"
Aug 04 09:38:18.318838 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.318800    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36"} err="failed to get container status \"e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36\": rpc error: code = NotFound desc = could not find container \"e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36\": container with ID starting with e3297bac32385e40d8a8268dc0faac703d7d9af66a29a6e06cb74fc6c7af6f36 not found: ID does not exist"
Aug 04 09:38:18.318938 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.318843    2558 scope.go:117] "RemoveContainer" containerID="7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86"
Aug 04 09:38:18.328573 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.328549    2558 scope.go:117] "RemoveContainer" containerID="7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86"
Aug 04 09:38:18.328931 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:38:18.328910    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86\": container with ID starting with 7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86 not found: ID does not exist" containerID="7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86"
Aug 04 09:38:18.329033 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.328941    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86"} err="failed to get container status \"7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86\": rpc error: code = NotFound desc = could not find container \"7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86\": container with ID starting with 7d8e362f93b89cd7e71e8ffed3333fcc2232c965def1120685853e60e7e95e86 not found: ID does not exist"
Aug 04 09:38:18.369936 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.369907    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c570a6-c4dc-4224-a258-f87df16acec2-plugins-config-volume\") pod \"e2c570a6-c4dc-4224-a258-f87df16acec2\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") "
Aug 04 09:38:18.370050 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.369965    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c69c2b-0159-407f-bc94-178411e1657b-plugins-config-volume\") pod \"98c69c2b-0159-407f-bc94-178411e1657b\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") "
Aug 04 09:38:18.370050 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.369993    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fllr\" (UniqueName: \"kubernetes.io/projected/98c69c2b-0159-407f-bc94-178411e1657b-kube-api-access-4fllr\") pod \"98c69c2b-0159-407f-bc94-178411e1657b\" (UID: \"98c69c2b-0159-407f-bc94-178411e1657b\") "
Aug 04 09:38:18.370162 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.370053    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl6bk\" (UniqueName: \"kubernetes.io/projected/e2c570a6-c4dc-4224-a258-f87df16acec2-kube-api-access-vl6bk\") pod \"e2c570a6-c4dc-4224-a258-f87df16acec2\" (UID: \"e2c570a6-c4dc-4224-a258-f87df16acec2\") "
Aug 04 09:38:18.370341 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.370311    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2c570a6-c4dc-4224-a258-f87df16acec2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2c570a6-c4dc-4224-a258-f87df16acec2" (UID: "e2c570a6-c4dc-4224-a258-f87df16acec2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:38:18.370418 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.370384    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c69c2b-0159-407f-bc94-178411e1657b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98c69c2b-0159-407f-bc94-178411e1657b" (UID: "98c69c2b-0159-407f-bc94-178411e1657b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:38:18.372196 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.372171    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c570a6-c4dc-4224-a258-f87df16acec2-kube-api-access-vl6bk" (OuterVolumeSpecName: "kube-api-access-vl6bk") pod "e2c570a6-c4dc-4224-a258-f87df16acec2" (UID: "e2c570a6-c4dc-4224-a258-f87df16acec2"). InnerVolumeSpecName "kube-api-access-vl6bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:38:18.372288 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.372257    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c69c2b-0159-407f-bc94-178411e1657b-kube-api-access-4fllr" (OuterVolumeSpecName: "kube-api-access-4fllr") pod "98c69c2b-0159-407f-bc94-178411e1657b" (UID: "98c69c2b-0159-407f-bc94-178411e1657b"). InnerVolumeSpecName "kube-api-access-4fllr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:38:18.471380 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.471340    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c69c2b-0159-407f-bc94-178411e1657b-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:38:18.471380 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.471372    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fllr\" (UniqueName: \"kubernetes.io/projected/98c69c2b-0159-407f-bc94-178411e1657b-kube-api-access-4fllr\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:38:18.471380 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.471382    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl6bk\" (UniqueName: \"kubernetes.io/projected/e2c570a6-c4dc-4224-a258-f87df16acec2-kube-api-access-vl6bk\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:38:18.471592 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.471391    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c570a6-c4dc-4224-a258-f87df16acec2-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:38:18.636885 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.636858    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"]
Aug 04 09:38:18.642790 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.642764    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-93164817-5d8cd6c465-x6szg"]
Aug 04 09:38:18.653701 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.653675    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"]
Aug 04 09:38:18.658303 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:18.658283    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-93164817-6d4f6c4748-tpx4g"]
Aug 04 09:38:19.212626 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.212597    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"]
Aug 04 09:38:19.213139 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213120    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c69c2b-0159-407f-bc94-178411e1657b" containerName="payload-pre-processing"
Aug 04 09:38:19.213208 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213143    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c69c2b-0159-407f-bc94-178411e1657b" containerName="payload-pre-processing"
Aug 04 09:38:19.213208 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213160    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c570a6-c4dc-4224-a258-f87df16acec2" containerName="payload-processing"
Aug 04 09:38:19.213208 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213170    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c570a6-c4dc-4224-a258-f87df16acec2" containerName="payload-processing"
Aug 04 09:38:19.213208 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213196    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d9f7247-fecb-49f2-bce8-57bfb0141b8c" containerName="revoke-keys"
Aug 04 09:38:19.213402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213220    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d9f7247-fecb-49f2-bce8-57bfb0141b8c" containerName="revoke-keys"
Aug 04 09:38:19.213402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213306    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2c570a6-c4dc-4224-a258-f87df16acec2" containerName="payload-processing"
Aug 04 09:38:19.213402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213320    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c69c2b-0159-407f-bc94-178411e1657b" containerName="payload-pre-processing"
Aug 04 09:38:19.213402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.213331    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d9f7247-fecb-49f2-bce8-57bfb0141b8c" containerName="revoke-keys"
Aug 04 09:38:19.217583 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.217563    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.220042 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.220024    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d6e1c0f5-openshift-default-dockercfg-lg7kg\""
Aug 04 09:38:19.229127 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.229105    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"]
Aug 04 09:38:19.278503 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-certs\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.278637 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-envoy\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.278637 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278543    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-token\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.278637 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-credential-socket\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.278876 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278636    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwvlm\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-kube-api-access-zwvlm\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.278876 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278723    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-podinfo\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.278876 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-data\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.278876 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istiod-ca-cert\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.279074 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.278879    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-socket\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.296916 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.296883    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d9f7247-fecb-49f2-bce8-57bfb0141b8c" path="/var/lib/kubelet/pods/5d9f7247-fecb-49f2-bce8-57bfb0141b8c/volumes"
Aug 04 09:38:19.297335 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.297310    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c69c2b-0159-407f-bc94-178411e1657b" path="/var/lib/kubelet/pods/98c69c2b-0159-407f-bc94-178411e1657b/volumes"
Aug 04 09:38:19.297841 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.297820    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2c570a6-c4dc-4224-a258-f87df16acec2" path="/var/lib/kubelet/pods/e2c570a6-c4dc-4224-a258-f87df16acec2/volumes"
Aug 04 09:38:19.380064 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-data\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380064 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istiod-ca-cert\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380275 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380133    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-socket\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380275 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-certs\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380275 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-envoy\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380427 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-token\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380427 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380409    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-credential-socket\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380523 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwvlm\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-kube-api-access-zwvlm\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380523 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-podinfo\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380621 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-certs\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380674 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-socket\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.380999 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.380968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-data\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.381103 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.381017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-credential-socket\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.381103 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.381088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istiod-ca-cert\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.382576 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.382557    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-envoy\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.383034 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.383018    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-podinfo\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.388814 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.388787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-token\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.388950 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.388934    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwvlm\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-kube-api-access-zwvlm\") pod \"e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") " pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.528316 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.528246    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:19.670129 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.670104    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"]
Aug 04 09:38:19.672246 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:38:19.672216    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f5fed8_d2d7_4ebc_86a0_206a3140ccd7.slice/crio-5a33d194c7ea3514ce3a5d9f51315484bb416a48c13d307857fe11823b656118 WatchSource:0}: Error finding container 5a33d194c7ea3514ce3a5d9f51315484bb416a48c13d307857fe11823b656118: Status 404 returned error can't find the container with id 5a33d194c7ea3514ce3a5d9f51315484bb416a48c13d307857fe11823b656118
Aug 04 09:38:19.674088 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.674059    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:38:19.674157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.674118    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:38:19.674157 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:19.674145    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:38:20.325377 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:20.325339    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" event={"ID":"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7","Type":"ContainerStarted","Data":"34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd"}
Aug 04 09:38:20.325750 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:20.325421    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" event={"ID":"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7","Type":"ContainerStarted","Data":"5a33d194c7ea3514ce3a5d9f51315484bb416a48c13d307857fe11823b656118"}
Aug 04 09:38:20.347395 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:20.347353    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podStartSLOduration=1.347337238 podStartE2EDuration="1.347337238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:38:20.346552196 +0000 UTC m=+1831.604509250" watchObservedRunningTime="2026-08-04 09:38:20.347337238 +0000 UTC m=+1831.605294359"
Aug 04 09:38:20.529069 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:20.529035    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:20.533767 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:20.533727    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:20.932944 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:20.932906    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:38:21.330982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:21.330955    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:21.331987 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:21.331969    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:29.144479 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:29.144433    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:38:35.395212 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:35.395176    2558 generic.go:356] "Generic (PLEG): container finished" podID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerID="34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd" exitCode=0
Aug 04 09:38:35.395603 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:35.395254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" event={"ID":"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7","Type":"ContainerDied","Data":"34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd"}
Aug 04 09:38:35.395603 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:35.395597    2558 scope.go:117] "RemoveContainer" containerID="34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd"
Aug 04 09:38:35.396547 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:35.396524    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:38:35.396626 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:35.396585    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:38:35.396626 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:35.396616    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:38:36.331811 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:36.331774    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:36.402156 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:36.402125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" event={"ID":"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7","Type":"ContainerStarted","Data":"6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530"}
Aug 04 09:38:36.530059 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:36.529957    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:36.530681 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:36.530370    2558 patch_prober.go:28] interesting pod/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Aug 04 09:38:36.530681 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:36.530421    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Aug 04 09:38:37.529090 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:37.529049    2558 patch_prober.go:28] interesting pod/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Aug 04 09:38:37.529560 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:37.529118    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Aug 04 09:38:38.528991 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:38.528951    2558 patch_prober.go:28] interesting pod/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Aug 04 09:38:38.529287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:38.529254    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Aug 04 09:38:39.529621 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:39.529583    2558 patch_prober.go:28] interesting pod/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Aug 04 09:38:39.530242 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:39.530200    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Aug 04 09:38:40.530071 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:40.529631    2558 patch_prober.go:28] interesting pod/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Aug 04 09:38:40.530071 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:40.529690    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Aug 04 09:38:41.533002 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:41.532964    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:41.533455 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:41.533367    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:41.534063 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:41.534043    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:38:55.026945 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:55.026912    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"]
Aug 04 09:38:55.027361 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:55.027159    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy" containerID="cri-o://6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530" gracePeriod=30
Aug 04 09:38:56.435202 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:56.435165    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:38:56.536374 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:56.536347    2558 patch_prober.go:28] interesting pod/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 09:38:56.536546 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:38:56.536413    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 09:39:00.289041 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.288987    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:39:00.353644 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353610    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-envoy\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.353857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353650    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-certs\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.353857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353690    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwvlm\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-kube-api-access-zwvlm\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.353857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353800    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-token\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.353857 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353828    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-credential-socket\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.354075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353903    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-data\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.354075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353932    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-socket\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.354075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353967    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-podinfo\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.354075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353974    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:39:00.354075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.353993    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istiod-ca-cert\") pod \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\" (UID: \"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7\") "
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354148    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354160    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-data" (OuterVolumeSpecName: "istio-data") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354217    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354350    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354368    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-credential-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354382    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-data\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354395    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-workload-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.354596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.354449    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:39:00.356200 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.356175    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-kube-api-access-zwvlm" (OuterVolumeSpecName: "kube-api-access-zwvlm") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "kube-api-access-zwvlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:39:00.356200 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.356186    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:39:00.356785 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.356763    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:39:00.356785 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.356772    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" (UID: "76f5fed8-d2d7-4ebc-86a0-206a3140ccd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 09:39:00.455710 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.455676    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-podinfo\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.455710 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.455704    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istiod-ca-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.455710 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.455714    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-envoy\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.455975 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.455722    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwvlm\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-kube-api-access-zwvlm\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.455975 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.455755    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7-istio-token\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:00.484782 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.484744    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"]
Aug 04 09:39:00.485156 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.485139    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy"
Aug 04 09:39:00.485239 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.485159    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy"
Aug 04 09:39:00.485300 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.485238    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy"
Aug 04 09:39:00.488450 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.488430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.491110 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.491075    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-cf8d7038\""
Aug 04 09:39:00.502914 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.502884    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"]
Aug 04 09:39:00.503392 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.503375    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy"
Aug 04 09:39:00.503477 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.503394    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy"
Aug 04 09:39:00.503536 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.503476    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerName="istio-proxy"
Aug 04 09:39:00.506500 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.506481    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"]
Aug 04 09:39:00.506606 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.506579    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.509220 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.509107    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-cf8d7038-dockercfg-g7qmv\""
Aug 04 09:39:00.512349 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.512326    2558 generic.go:356] "Generic (PLEG): container finished" podID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" containerID="6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530" exitCode=0
Aug 04 09:39:00.512458 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.512378    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"
Aug 04 09:39:00.512458 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.512391    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" event={"ID":"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7","Type":"ContainerDied","Data":"6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530"}
Aug 04 09:39:00.512458 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.512435    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4" event={"ID":"76f5fed8-d2d7-4ebc-86a0-206a3140ccd7","Type":"ContainerDied","Data":"5a33d194c7ea3514ce3a5d9f51315484bb416a48c13d307857fe11823b656118"}
Aug 04 09:39:00.512458 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.512458    2558 scope.go:117] "RemoveContainer" containerID="6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530"
Aug 04 09:39:00.520386 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.520328    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"]
Aug 04 09:39:00.526093 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.526070    2558 scope.go:117] "RemoveContainer" containerID="34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd"
Aug 04 09:39:00.534590 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.534569    2558 scope.go:117] "RemoveContainer" containerID="6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530"
Aug 04 09:39:00.534937 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:39:00.534916    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530\": container with ID starting with 6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530 not found: ID does not exist" containerID="6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530"
Aug 04 09:39:00.535018 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.534948    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530"} err="failed to get container status \"6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530\": rpc error: code = NotFound desc = could not find container \"6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530\": container with ID starting with 6f7c0df3905eac8462fe0a5fa10686eafeeb0754586c8b163400a3c4bcd4e530 not found: ID does not exist"
Aug 04 09:39:00.535018 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.534966    2558 scope.go:117] "RemoveContainer" containerID="34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd"
Aug 04 09:39:00.535253 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:39:00.535232    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd\": container with ID starting with 34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd not found: ID does not exist" containerID="34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd"
Aug 04 09:39:00.535292 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.535262    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd"} err="failed to get container status \"34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd\": rpc error: code = NotFound desc = could not find container \"34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd\": container with ID starting with 34f4c2290e61a1729f3e121b52b4bf0bde77b8009971d5d4ecabec2a633108cd not found: ID does not exist"
Aug 04 09:39:00.549073 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.549020    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"]
Aug 04 09:39:00.553162 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.553138    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d6e1c0f5-openshift-default-74769c9976-j9md4"]
Aug 04 09:39:00.556973 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.556952    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973d33ac-2f26-4f53-bfce-3d035c0cc74b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.557083 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.556997    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqj4k\" (UniqueName: \"kubernetes.io/projected/973d33ac-2f26-4f53-bfce-3d035c0cc74b-kube-api-access-zqj4k\") pod \"payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.557083 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.557052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18628c9-670b-4d7b-ad6a-a129c2d453da-plugins-config-volume\") pod \"payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") " pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.557196 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.557080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7r5l\" (UniqueName: \"kubernetes.io/projected/f18628c9-670b-4d7b-ad6a-a129c2d453da-kube-api-access-p7r5l\") pod \"payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") " pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.658329 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.658295    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973d33ac-2f26-4f53-bfce-3d035c0cc74b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.658493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.658342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqj4k\" (UniqueName: \"kubernetes.io/projected/973d33ac-2f26-4f53-bfce-3d035c0cc74b-kube-api-access-zqj4k\") pod \"payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.658493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.658473    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18628c9-670b-4d7b-ad6a-a129c2d453da-plugins-config-volume\") pod \"payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") " pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.658578 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.658519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7r5l\" (UniqueName: \"kubernetes.io/projected/f18628c9-670b-4d7b-ad6a-a129c2d453da-kube-api-access-p7r5l\") pod \"payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") " pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.659303 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.659273    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973d33ac-2f26-4f53-bfce-3d035c0cc74b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.659403 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.659372    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18628c9-670b-4d7b-ad6a-a129c2d453da-plugins-config-volume\") pod \"payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") " pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.666826 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.666803    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqj4k\" (UniqueName: \"kubernetes.io/projected/973d33ac-2f26-4f53-bfce-3d035c0cc74b-kube-api-access-zqj4k\") pod \"payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.666923 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.666835    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7r5l\" (UniqueName: \"kubernetes.io/projected/f18628c9-670b-4d7b-ad6a-a129c2d453da-kube-api-access-p7r5l\") pod \"payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") " pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.798464 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.798432    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:00.820354 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.820328    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:00.952809 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:00.952784    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"]
Aug 04 09:39:00.954960 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:39:00.954918    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973d33ac_2f26_4f53_bfce_3d035c0cc74b.slice/crio-687da662d72bdebe351038affc71319b451131fabc09c6417af7add4a87e87be WatchSource:0}: Error finding container 687da662d72bdebe351038affc71319b451131fabc09c6417af7add4a87e87be: Status 404 returned error can't find the container with id 687da662d72bdebe351038affc71319b451131fabc09c6417af7add4a87e87be
Aug 04 09:39:01.179848 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.179825    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"]
Aug 04 09:39:01.182290 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:39:01.182262    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf18628c9_670b_4d7b_ad6a_a129c2d453da.slice/crio-362d379d84e2754e53d35b8636e3eea942299ce5dc980eac143674d5f9e233cb WatchSource:0}: Error finding container 362d379d84e2754e53d35b8636e3eea942299ce5dc980eac143674d5f9e233cb: Status 404 returned error can't find the container with id 362d379d84e2754e53d35b8636e3eea942299ce5dc980eac143674d5f9e233cb
Aug 04 09:39:01.300479 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.300440    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f5fed8-d2d7-4ebc-86a0-206a3140ccd7" path="/var/lib/kubelet/pods/76f5fed8-d2d7-4ebc-86a0-206a3140ccd7/volumes"
Aug 04 09:39:01.518451 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.518346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp" event={"ID":"f18628c9-670b-4d7b-ad6a-a129c2d453da","Type":"ContainerStarted","Data":"8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000"}
Aug 04 09:39:01.518617 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.518536    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:01.518617 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.518602    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp" event={"ID":"f18628c9-670b-4d7b-ad6a-a129c2d453da","Type":"ContainerStarted","Data":"362d379d84e2754e53d35b8636e3eea942299ce5dc980eac143674d5f9e233cb"}
Aug 04 09:39:01.520860 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.520834    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp" event={"ID":"973d33ac-2f26-4f53-bfce-3d035c0cc74b","Type":"ContainerStarted","Data":"6c2f150ae657a3e39e0c6cd877586d2d90e99f03fb0caa1601df761d6bc895b8"}
Aug 04 09:39:01.521008 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.520881    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp" event={"ID":"973d33ac-2f26-4f53-bfce-3d035c0cc74b","Type":"ContainerStarted","Data":"687da662d72bdebe351038affc71319b451131fabc09c6417af7add4a87e87be"}
Aug 04 09:39:01.521008 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.520929    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:01.539760 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.539696    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp" podStartSLOduration=1.539680304 podStartE2EDuration="1.539680304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:39:01.537059709 +0000 UTC m=+1872.795016764" watchObservedRunningTime="2026-08-04 09:39:01.539680304 +0000 UTC m=+1872.797637360"
Aug 04 09:39:01.557346 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:01.557292    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp" podStartSLOduration=1.557276337 podStartE2EDuration="1.557276337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:39:01.555343017 +0000 UTC m=+1872.813300073" watchObservedRunningTime="2026-08-04 09:39:01.557276337 +0000 UTC m=+1872.815233390"
Aug 04 09:39:02.750970 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:02.750934    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:39:12.528461 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:12.528387    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:12.528461 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:12.528452    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:23.681390 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.681358    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"]
Aug 04 09:39:23.684582 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.684563    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"
Aug 04 09:39:23.686928 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.686909    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\""
Aug 04 09:39:23.691371 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.691350    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"]
Aug 04 09:39:23.859776 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.859715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/352f9546-aca6-4578-bbfa-e1a9b1cf26d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76\" (UID: \"352f9546-aca6-4578-bbfa-e1a9b1cf26d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"
Aug 04 09:39:23.960482 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.960413    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/352f9546-aca6-4578-bbfa-e1a9b1cf26d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76\" (UID: \"352f9546-aca6-4578-bbfa-e1a9b1cf26d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"
Aug 04 09:39:23.961000 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.960982    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/352f9546-aca6-4578-bbfa-e1a9b1cf26d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76\" (UID: \"352f9546-aca6-4578-bbfa-e1a9b1cf26d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"
Aug 04 09:39:23.996856 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:23.996826    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"
Aug 04 09:39:24.146132 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:24.146095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"]
Aug 04 09:39:24.149145 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:39:24.149106    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352f9546_aca6_4578_bbfa_e1a9b1cf26d2.slice/crio-8959efd949165c1ab6f4e20fa9cd4ad5371eb19472200206bf110eaa3c434bec WatchSource:0}: Error finding container 8959efd949165c1ab6f4e20fa9cd4ad5371eb19472200206bf110eaa3c434bec: Status 404 returned error can't find the container with id 8959efd949165c1ab6f4e20fa9cd4ad5371eb19472200206bf110eaa3c434bec
Aug 04 09:39:24.631831 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:24.631727    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76" event={"ID":"352f9546-aca6-4578-bbfa-e1a9b1cf26d2","Type":"ContainerStarted","Data":"388b2fb5c1658e3f467781c41bb4cdca2bae5e3316be10debed036bf9512ef91"}
Aug 04 09:39:24.631831 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:24.631802    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76" event={"ID":"352f9546-aca6-4578-bbfa-e1a9b1cf26d2","Type":"ContainerStarted","Data":"8959efd949165c1ab6f4e20fa9cd4ad5371eb19472200206bf110eaa3c434bec"}
Aug 04 09:39:24.648672 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:24.648620    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76" podStartSLOduration=1.6486036720000001 podStartE2EDuration="1.648603672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:39:24.648062937 +0000 UTC m=+1895.906019992" watchObservedRunningTime="2026-08-04 09:39:24.648603672 +0000 UTC m=+1895.906560727"
Aug 04 09:39:25.637835 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:25.637798    2558 generic.go:356] "Generic (PLEG): container finished" podID="352f9546-aca6-4578-bbfa-e1a9b1cf26d2" containerID="388b2fb5c1658e3f467781c41bb4cdca2bae5e3316be10debed036bf9512ef91" exitCode=0
Aug 04 09:39:25.638214 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:25.637905    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76" event={"ID":"352f9546-aca6-4578-bbfa-e1a9b1cf26d2","Type":"ContainerDied","Data":"388b2fb5c1658e3f467781c41bb4cdca2bae5e3316be10debed036bf9512ef91"}
Aug 04 09:39:26.776840 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:26.776811    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"
Aug 04 09:39:26.888281 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:26.888248    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/352f9546-aca6-4578-bbfa-e1a9b1cf26d2-maas-api-service-ca\") pod \"352f9546-aca6-4578-bbfa-e1a9b1cf26d2\" (UID: \"352f9546-aca6-4578-bbfa-e1a9b1cf26d2\") "
Aug 04 09:39:26.888780 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:26.888722    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/352f9546-aca6-4578-bbfa-e1a9b1cf26d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "352f9546-aca6-4578-bbfa-e1a9b1cf26d2" (UID: "352f9546-aca6-4578-bbfa-e1a9b1cf26d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:39:26.989543 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:26.989453    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/352f9546-aca6-4578-bbfa-e1a9b1cf26d2-maas-api-service-ca\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:27.648838 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:27.648799    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76" event={"ID":"352f9546-aca6-4578-bbfa-e1a9b1cf26d2","Type":"ContainerDied","Data":"8959efd949165c1ab6f4e20fa9cd4ad5371eb19472200206bf110eaa3c434bec"}
Aug 04 09:39:27.648838 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:27.648821    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"
Aug 04 09:39:27.649030 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:27.648857    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8959efd949165c1ab6f4e20fa9cd4ad5371eb19472200206bf110eaa3c434bec"
Aug 04 09:39:30.369558 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.369526    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"]
Aug 04 09:39:30.370076 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.369755    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp" podUID="f18628c9-670b-4d7b-ad6a-a129c2d453da" containerName="payload-processing" containerID="cri-o://8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000" gracePeriod=30
Aug 04 09:39:30.379600 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.379045    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"]
Aug 04 09:39:30.379600 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.379262    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp" podUID="973d33ac-2f26-4f53-bfce-3d035c0cc74b" containerName="payload-pre-processing" containerID="cri-o://6c2f150ae657a3e39e0c6cd877586d2d90e99f03fb0caa1601df761d6bc895b8" gracePeriod=30
Aug 04 09:39:30.515823 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.515781    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"]
Aug 04 09:39:30.518408 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.518368    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cf8d7038-hfh76"]
Aug 04 09:39:30.663155 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.663129    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:30.667081 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.667050    2558 generic.go:356] "Generic (PLEG): container finished" podID="973d33ac-2f26-4f53-bfce-3d035c0cc74b" containerID="6c2f150ae657a3e39e0c6cd877586d2d90e99f03fb0caa1601df761d6bc895b8" exitCode=0
Aug 04 09:39:30.667212 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.667138    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp" event={"ID":"973d33ac-2f26-4f53-bfce-3d035c0cc74b","Type":"ContainerDied","Data":"6c2f150ae657a3e39e0c6cd877586d2d90e99f03fb0caa1601df761d6bc895b8"}
Aug 04 09:39:30.667304 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.667229    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp" event={"ID":"973d33ac-2f26-4f53-bfce-3d035c0cc74b","Type":"ContainerDied","Data":"687da662d72bdebe351038affc71319b451131fabc09c6417af7add4a87e87be"}
Aug 04 09:39:30.667304 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.667254    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="687da662d72bdebe351038affc71319b451131fabc09c6417af7add4a87e87be"
Aug 04 09:39:30.667405 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.667389    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:30.668690 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.668666    2558 generic.go:356] "Generic (PLEG): container finished" podID="f18628c9-670b-4d7b-ad6a-a129c2d453da" containerID="8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000" exitCode=0
Aug 04 09:39:30.668839 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.668713    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"
Aug 04 09:39:30.668839 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.668770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp" event={"ID":"f18628c9-670b-4d7b-ad6a-a129c2d453da","Type":"ContainerDied","Data":"8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000"}
Aug 04 09:39:30.668839 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.668822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp" event={"ID":"f18628c9-670b-4d7b-ad6a-a129c2d453da","Type":"ContainerDied","Data":"362d379d84e2754e53d35b8636e3eea942299ce5dc980eac143674d5f9e233cb"}
Aug 04 09:39:30.669004 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.668853    2558 scope.go:117] "RemoveContainer" containerID="8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000"
Aug 04 09:39:30.678452 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.678431    2558 scope.go:117] "RemoveContainer" containerID="8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000"
Aug 04 09:39:30.678714 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:39:30.678696    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000\": container with ID starting with 8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000 not found: ID does not exist" containerID="8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000"
Aug 04 09:39:30.678849 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.678723    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000"} err="failed to get container status \"8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000\": rpc error: code = NotFound desc = could not find container \"8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000\": container with ID starting with 8bbaf5c3969b137bf267ce48d9504d64cd17d918c969af937f570de38351a000 not found: ID does not exist"
Aug 04 09:39:30.721094 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.721069    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18628c9-670b-4d7b-ad6a-a129c2d453da-plugins-config-volume\") pod \"f18628c9-670b-4d7b-ad6a-a129c2d453da\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") "
Aug 04 09:39:30.721255 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.721106    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973d33ac-2f26-4f53-bfce-3d035c0cc74b-plugins-config-volume\") pod \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") "
Aug 04 09:39:30.721255 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.721133    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqj4k\" (UniqueName: \"kubernetes.io/projected/973d33ac-2f26-4f53-bfce-3d035c0cc74b-kube-api-access-zqj4k\") pod \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\" (UID: \"973d33ac-2f26-4f53-bfce-3d035c0cc74b\") "
Aug 04 09:39:30.721255 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.721156    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7r5l\" (UniqueName: \"kubernetes.io/projected/f18628c9-670b-4d7b-ad6a-a129c2d453da-kube-api-access-p7r5l\") pod \"f18628c9-670b-4d7b-ad6a-a129c2d453da\" (UID: \"f18628c9-670b-4d7b-ad6a-a129c2d453da\") "
Aug 04 09:39:30.721474 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.721449    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f18628c9-670b-4d7b-ad6a-a129c2d453da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f18628c9-670b-4d7b-ad6a-a129c2d453da" (UID: "f18628c9-670b-4d7b-ad6a-a129c2d453da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:39:30.721580 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.721476    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/973d33ac-2f26-4f53-bfce-3d035c0cc74b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "973d33ac-2f26-4f53-bfce-3d035c0cc74b" (UID: "973d33ac-2f26-4f53-bfce-3d035c0cc74b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:39:30.723411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.723386    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f18628c9-670b-4d7b-ad6a-a129c2d453da-kube-api-access-p7r5l" (OuterVolumeSpecName: "kube-api-access-p7r5l") pod "f18628c9-670b-4d7b-ad6a-a129c2d453da" (UID: "f18628c9-670b-4d7b-ad6a-a129c2d453da"). InnerVolumeSpecName "kube-api-access-p7r5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:39:30.723411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.723399    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/973d33ac-2f26-4f53-bfce-3d035c0cc74b-kube-api-access-zqj4k" (OuterVolumeSpecName: "kube-api-access-zqj4k") pod "973d33ac-2f26-4f53-bfce-3d035c0cc74b" (UID: "973d33ac-2f26-4f53-bfce-3d035c0cc74b"). InnerVolumeSpecName "kube-api-access-zqj4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:39:30.822275 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.822245    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18628c9-670b-4d7b-ad6a-a129c2d453da-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:30.822275 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.822274    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/973d33ac-2f26-4f53-bfce-3d035c0cc74b-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:30.822476 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.822283    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqj4k\" (UniqueName: \"kubernetes.io/projected/973d33ac-2f26-4f53-bfce-3d035c0cc74b-kube-api-access-zqj4k\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:30.822476 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.822293    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7r5l\" (UniqueName: \"kubernetes.io/projected/f18628c9-670b-4d7b-ad6a-a129c2d453da-kube-api-access-p7r5l\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:39:30.992483 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.992449    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"]
Aug 04 09:39:30.995970 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:30.995938    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cf8d7038-694ff84646-7fwfp"]
Aug 04 09:39:31.295577 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:31.295483    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="352f9546-aca6-4578-bbfa-e1a9b1cf26d2" path="/var/lib/kubelet/pods/352f9546-aca6-4578-bbfa-e1a9b1cf26d2/volumes"
Aug 04 09:39:31.295955 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:31.295936    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f18628c9-670b-4d7b-ad6a-a129c2d453da" path="/var/lib/kubelet/pods/f18628c9-670b-4d7b-ad6a-a129c2d453da/volumes"
Aug 04 09:39:31.675105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:31.675077    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"
Aug 04 09:39:31.691242 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:31.691214    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"]
Aug 04 09:39:31.699317 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:31.699278    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cf8d7038-646cd4b4b4-nsrpp"]
Aug 04 09:39:32.447761 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:32.447706    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:39:33.296422 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:33.296377    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="973d33ac-2f26-4f53-bfce-3d035c0cc74b" path="/var/lib/kubelet/pods/973d33ac-2f26-4f53-bfce-3d035c0cc74b/volumes"
Aug 04 09:39:39.746303 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:39.746267    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:39:45.530428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:39:45.530394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:41:20.640978 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:20.640888    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:41:25.532620 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:25.532582    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:41:35.052085 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052050    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"]
Aug 04 09:41:35.052446 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052414    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="352f9546-aca6-4578-bbfa-e1a9b1cf26d2" containerName="revoke-keys"
Aug 04 09:41:35.052446 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052425    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="352f9546-aca6-4578-bbfa-e1a9b1cf26d2" containerName="revoke-keys"
Aug 04 09:41:35.052446 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052435    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f18628c9-670b-4d7b-ad6a-a129c2d453da" containerName="payload-processing"
Aug 04 09:41:35.052446 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052441    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f18628c9-670b-4d7b-ad6a-a129c2d453da" containerName="payload-processing"
Aug 04 09:41:35.052575 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052450    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="973d33ac-2f26-4f53-bfce-3d035c0cc74b" containerName="payload-pre-processing"
Aug 04 09:41:35.052575 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052456    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="973d33ac-2f26-4f53-bfce-3d035c0cc74b" containerName="payload-pre-processing"
Aug 04 09:41:35.052575 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052512    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f18628c9-670b-4d7b-ad6a-a129c2d453da" containerName="payload-processing"
Aug 04 09:41:35.052575 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052520    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="973d33ac-2f26-4f53-bfce-3d035c0cc74b" containerName="payload-pre-processing"
Aug 04 09:41:35.052575 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.052526    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="352f9546-aca6-4578-bbfa-e1a9b1cf26d2" containerName="revoke-keys"
Aug 04 09:41:35.055410 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.055392    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.058410 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.058387    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ea2eafd7-dockercfg-cwrsv\""
Aug 04 09:41:35.058550 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.058397    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ea2eafd7\""
Aug 04 09:41:35.065170 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.065150    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"]
Aug 04 09:41:35.168661 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.168621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29d9q\" (UniqueName: \"kubernetes.io/projected/7eb8db20-d8af-47f8-a38e-84136b5670e8-kube-api-access-29d9q\") pod \"payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.168883 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.168768    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb8db20-d8af-47f8-a38e-84136b5670e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.270022 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.269988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb8db20-d8af-47f8-a38e-84136b5670e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.270202 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.270074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29d9q\" (UniqueName: \"kubernetes.io/projected/7eb8db20-d8af-47f8-a38e-84136b5670e8-kube-api-access-29d9q\") pod \"payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.270786 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.270725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb8db20-d8af-47f8-a38e-84136b5670e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.278872 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.278846    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29d9q\" (UniqueName: \"kubernetes.io/projected/7eb8db20-d8af-47f8-a38e-84136b5670e8-kube-api-access-29d9q\") pod \"payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.365981 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.365949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:35.504474 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:35.504280    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"]
Aug 04 09:41:35.507350 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:41:35.507325    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb8db20_d8af_47f8_a38e_84136b5670e8.slice/crio-ddf7fef378c2233c444ee5ceb51a5df0e61eca6811f3bbd0273301f6bfd05a5b WatchSource:0}: Error finding container ddf7fef378c2233c444ee5ceb51a5df0e61eca6811f3bbd0273301f6bfd05a5b: Status 404 returned error can't find the container with id ddf7fef378c2233c444ee5ceb51a5df0e61eca6811f3bbd0273301f6bfd05a5b
Aug 04 09:41:36.256844 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:36.256804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2" event={"ID":"7eb8db20-d8af-47f8-a38e-84136b5670e8","Type":"ContainerStarted","Data":"dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7"}
Aug 04 09:41:36.257251 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:36.256871    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2" event={"ID":"7eb8db20-d8af-47f8-a38e-84136b5670e8","Type":"ContainerStarted","Data":"ddf7fef378c2233c444ee5ceb51a5df0e61eca6811f3bbd0273301f6bfd05a5b"}
Aug 04 09:41:36.257251 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:36.256909    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:41:37.127632 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:37.127477    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2" podStartSLOduration=2.127452942 podStartE2EDuration="2.127452942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:36.272921199 +0000 UTC m=+2027.530878253" watchObservedRunningTime="2026-08-04 09:41:37.127452942 +0000 UTC m=+2028.385409997"
Aug 04 09:41:37.129253 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:37.129226    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:41:47.264314 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:41:47.264287    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:42:05.680578 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.680542    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"]
Aug 04 09:42:05.684182 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.684163    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"
Aug 04 09:42:05.687302 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.687283    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\""
Aug 04 09:42:05.699359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.699336    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"]
Aug 04 09:42:05.835394 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.835358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e3a81a-79df-4a31-ac84-c1252543ef9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r\" (UID: \"18e3a81a-79df-4a31-ac84-c1252543ef9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"
Aug 04 09:42:05.936162 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.936063    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e3a81a-79df-4a31-ac84-c1252543ef9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r\" (UID: \"18e3a81a-79df-4a31-ac84-c1252543ef9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"
Aug 04 09:42:05.936725 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.936700    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e3a81a-79df-4a31-ac84-c1252543ef9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r\" (UID: \"18e3a81a-79df-4a31-ac84-c1252543ef9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"
Aug 04 09:42:05.994108 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:05.994078    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"
Aug 04 09:42:06.123749 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:06.123685    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"]
Aug 04 09:42:06.126368 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:42:06.126334    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e3a81a_79df_4a31_ac84_c1252543ef9d.slice/crio-767826529b4d72ab7b5424025c41a3bebaa1e5423797cd5ddc79ade37c08d08c WatchSource:0}: Error finding container 767826529b4d72ab7b5424025c41a3bebaa1e5423797cd5ddc79ade37c08d08c: Status 404 returned error can't find the container with id 767826529b4d72ab7b5424025c41a3bebaa1e5423797cd5ddc79ade37c08d08c
Aug 04 09:42:06.400562 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:06.400524    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r" event={"ID":"18e3a81a-79df-4a31-ac84-c1252543ef9d","Type":"ContainerStarted","Data":"767826529b4d72ab7b5424025c41a3bebaa1e5423797cd5ddc79ade37c08d08c"}
Aug 04 09:42:07.406846 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:07.406810    2558 generic.go:356] "Generic (PLEG): container finished" podID="18e3a81a-79df-4a31-ac84-c1252543ef9d" containerID="574b5af8e9663ed89e39adc5137e0ba70ca67fdf8cf3caf6d8a93339f9f486b7" exitCode=0
Aug 04 09:42:07.407215 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:07.406906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r" event={"ID":"18e3a81a-79df-4a31-ac84-c1252543ef9d","Type":"ContainerDied","Data":"574b5af8e9663ed89e39adc5137e0ba70ca67fdf8cf3caf6d8a93339f9f486b7"}
Aug 04 09:42:08.537474 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:08.537445    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"
Aug 04 09:42:08.658801 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:08.658769    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e3a81a-79df-4a31-ac84-c1252543ef9d-maas-api-service-ca\") pod \"18e3a81a-79df-4a31-ac84-c1252543ef9d\" (UID: \"18e3a81a-79df-4a31-ac84-c1252543ef9d\") "
Aug 04 09:42:08.659116 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:08.659093    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e3a81a-79df-4a31-ac84-c1252543ef9d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18e3a81a-79df-4a31-ac84-c1252543ef9d" (UID: "18e3a81a-79df-4a31-ac84-c1252543ef9d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:42:08.760279 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:08.760204    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e3a81a-79df-4a31-ac84-c1252543ef9d-maas-api-service-ca\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:42:09.417282 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:09.417254    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"
Aug 04 09:42:09.417428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:09.417288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r" event={"ID":"18e3a81a-79df-4a31-ac84-c1252543ef9d","Type":"ContainerDied","Data":"767826529b4d72ab7b5424025c41a3bebaa1e5423797cd5ddc79ade37c08d08c"}
Aug 04 09:42:09.417428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:09.417334    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="767826529b4d72ab7b5424025c41a3bebaa1e5423797cd5ddc79ade37c08d08c"
Aug 04 09:42:16.161902 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.161646    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"]
Aug 04 09:42:16.162579 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.162518    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2" podUID="7eb8db20-d8af-47f8-a38e-84136b5670e8" containerName="payload-pre-processing" containerID="cri-o://dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7" gracePeriod=30
Aug 04 09:42:16.294808 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.294780    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"]
Aug 04 09:42:16.299528 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.299497    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea2eafd7-dsd6r"]
Aug 04 09:42:16.412550 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.412496    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:42:16.426075 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.426023    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb8db20-d8af-47f8-a38e-84136b5670e8-plugins-config-volume\") pod \"7eb8db20-d8af-47f8-a38e-84136b5670e8\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") "
Aug 04 09:42:16.426227 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.426081    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29d9q\" (UniqueName: \"kubernetes.io/projected/7eb8db20-d8af-47f8-a38e-84136b5670e8-kube-api-access-29d9q\") pod \"7eb8db20-d8af-47f8-a38e-84136b5670e8\" (UID: \"7eb8db20-d8af-47f8-a38e-84136b5670e8\") "
Aug 04 09:42:16.426431 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.426408    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eb8db20-d8af-47f8-a38e-84136b5670e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7eb8db20-d8af-47f8-a38e-84136b5670e8" (UID: "7eb8db20-d8af-47f8-a38e-84136b5670e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:42:16.428368 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.428339    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb8db20-d8af-47f8-a38e-84136b5670e8-kube-api-access-29d9q" (OuterVolumeSpecName: "kube-api-access-29d9q") pod "7eb8db20-d8af-47f8-a38e-84136b5670e8" (UID: "7eb8db20-d8af-47f8-a38e-84136b5670e8"). InnerVolumeSpecName "kube-api-access-29d9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:42:16.450096 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.450066    2558 generic.go:356] "Generic (PLEG): container finished" podID="7eb8db20-d8af-47f8-a38e-84136b5670e8" containerID="dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7" exitCode=0
Aug 04 09:42:16.450199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.450110    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"
Aug 04 09:42:16.450199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.450140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2" event={"ID":"7eb8db20-d8af-47f8-a38e-84136b5670e8","Type":"ContainerDied","Data":"dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7"}
Aug 04 09:42:16.450199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.450190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2" event={"ID":"7eb8db20-d8af-47f8-a38e-84136b5670e8","Type":"ContainerDied","Data":"ddf7fef378c2233c444ee5ceb51a5df0e61eca6811f3bbd0273301f6bfd05a5b"}
Aug 04 09:42:16.450306 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.450213    2558 scope.go:117] "RemoveContainer" containerID="dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7"
Aug 04 09:42:16.459512 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.459497    2558 scope.go:117] "RemoveContainer" containerID="dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7"
Aug 04 09:42:16.459822 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:42:16.459718    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7\": container with ID starting with dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7 not found: ID does not exist" containerID="dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7"
Aug 04 09:42:16.459822 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.459769    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7"} err="failed to get container status \"dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7\": rpc error: code = NotFound desc = could not find container \"dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7\": container with ID starting with dc329a468cdc2862e4dffc710171df05a1bb122471849d5040ceb8f3be3139b7 not found: ID does not exist"
Aug 04 09:42:16.471257 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.471238    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"]
Aug 04 09:42:16.475105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.475083    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea2eafd7-5b658ff89b-mkts2"]
Aug 04 09:42:16.527261 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.527242    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb8db20-d8af-47f8-a38e-84136b5670e8-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:42:16.527261 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:16.527261    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29d9q\" (UniqueName: \"kubernetes.io/projected/7eb8db20-d8af-47f8-a38e-84136b5670e8-kube-api-access-29d9q\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:42:17.296059 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:17.296028    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e3a81a-79df-4a31-ac84-c1252543ef9d" path="/var/lib/kubelet/pods/18e3a81a-79df-4a31-ac84-c1252543ef9d/volumes"
Aug 04 09:42:17.296513 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:17.296376    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb8db20-d8af-47f8-a38e-84136b5670e8" path="/var/lib/kubelet/pods/7eb8db20-d8af-47f8-a38e-84136b5670e8/volumes"
Aug 04 09:42:18.333888 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:18.333855    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:42:57.028515 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:42:57.028480    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:44:41.931911 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:41.931873    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:44:44.362159 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.362118    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"]
Aug 04 09:44:44.362654 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.362632    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e3a81a-79df-4a31-ac84-c1252543ef9d" containerName="revoke-keys"
Aug 04 09:44:44.362796 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.362657    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e3a81a-79df-4a31-ac84-c1252543ef9d" containerName="revoke-keys"
Aug 04 09:44:44.362796 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.362700    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb8db20-d8af-47f8-a38e-84136b5670e8" containerName="payload-pre-processing"
Aug 04 09:44:44.362796 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.362710    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb8db20-d8af-47f8-a38e-84136b5670e8" containerName="payload-pre-processing"
Aug 04 09:44:44.362963 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.362825    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eb8db20-d8af-47f8-a38e-84136b5670e8" containerName="payload-pre-processing"
Aug 04 09:44:44.362963 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.362841    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e3a81a-79df-4a31-ac84-c1252543ef9d" containerName="revoke-keys"
Aug 04 09:44:44.366152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.366132    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.369821 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.369801    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-18d97d1d-dockercfg-5hrwf\""
Aug 04 09:44:44.370040 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.370024    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-18d97d1d\""
Aug 04 09:44:44.381124 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.381095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"]
Aug 04 09:44:44.415909 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.415879    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"]
Aug 04 09:44:44.419383 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.419362    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.430404 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.430373    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"]
Aug 04 09:44:44.466964 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.466935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0edc384-a78f-4cbe-ae20-5c113d7b5244-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.467110 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.466992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkblb\" (UniqueName: \"kubernetes.io/projected/e0edc384-a78f-4cbe-ae20-5c113d7b5244-kube-api-access-mkblb\") pod \"payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.568089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.568054    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0edc384-a78f-4cbe-ae20-5c113d7b5244-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.568276 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.568144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkblb\" (UniqueName: \"kubernetes.io/projected/e0edc384-a78f-4cbe-ae20-5c113d7b5244-kube-api-access-mkblb\") pod \"payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.568276 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.568183    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/157bd973-c8d4-4199-b27f-c5e3e73d4698-plugins-config-volume\") pod \"payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") " pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.568276 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.568260    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skb8h\" (UniqueName: \"kubernetes.io/projected/157bd973-c8d4-4199-b27f-c5e3e73d4698-kube-api-access-skb8h\") pod \"payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") " pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.568891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.568872    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0edc384-a78f-4cbe-ae20-5c113d7b5244-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.577158 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.577138    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkblb\" (UniqueName: \"kubernetes.io/projected/e0edc384-a78f-4cbe-ae20-5c113d7b5244-kube-api-access-mkblb\") pod \"payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.669074 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.669001    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skb8h\" (UniqueName: \"kubernetes.io/projected/157bd973-c8d4-4199-b27f-c5e3e73d4698-kube-api-access-skb8h\") pod \"payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") " pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.669074 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.669066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/157bd973-c8d4-4199-b27f-c5e3e73d4698-plugins-config-volume\") pod \"payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") " pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.669763 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.669727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/157bd973-c8d4-4199-b27f-c5e3e73d4698-plugins-config-volume\") pod \"payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") " pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.677833 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.677813    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skb8h\" (UniqueName: \"kubernetes.io/projected/157bd973-c8d4-4199-b27f-c5e3e73d4698-kube-api-access-skb8h\") pod \"payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") " pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.682612 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.682591    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:44.733965 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.733813    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:44.832522 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:44.832492    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"]
Aug 04 09:44:44.835536 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:44:44.835495    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0edc384_a78f_4cbe_ae20_5c113d7b5244.slice/crio-d7e49c6788e9d8a5cf91207646e01504302e3479f486cab91f48b6f7f3957ce7 WatchSource:0}: Error finding container d7e49c6788e9d8a5cf91207646e01504302e3479f486cab91f48b6f7f3957ce7: Status 404 returned error can't find the container with id d7e49c6788e9d8a5cf91207646e01504302e3479f486cab91f48b6f7f3957ce7
Aug 04 09:44:45.100845 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:45.100819    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"]
Aug 04 09:44:45.103278 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:44:45.103253    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157bd973_c8d4_4199_b27f_c5e3e73d4698.slice/crio-384e5c78bc14021c87275f6aafb0854f75e3aede681e166d71f91b73c14344d2 WatchSource:0}: Error finding container 384e5c78bc14021c87275f6aafb0854f75e3aede681e166d71f91b73c14344d2: Status 404 returned error can't find the container with id 384e5c78bc14021c87275f6aafb0854f75e3aede681e166d71f91b73c14344d2
Aug 04 09:44:45.140123 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:45.140029    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx" event={"ID":"e0edc384-a78f-4cbe-ae20-5c113d7b5244","Type":"ContainerStarted","Data":"e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445"}
Aug 04 09:44:45.140123 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:45.140093    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx" event={"ID":"e0edc384-a78f-4cbe-ae20-5c113d7b5244","Type":"ContainerStarted","Data":"d7e49c6788e9d8a5cf91207646e01504302e3479f486cab91f48b6f7f3957ce7"}
Aug 04 09:44:45.140312 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:45.140256    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:45.141344 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:45.141322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk" event={"ID":"157bd973-c8d4-4199-b27f-c5e3e73d4698","Type":"ContainerStarted","Data":"384e5c78bc14021c87275f6aafb0854f75e3aede681e166d71f91b73c14344d2"}
Aug 04 09:44:45.159639 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:45.159529    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx" podStartSLOduration=1.159509873 podStartE2EDuration="1.159509873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:44:45.156258717 +0000 UTC m=+2216.414215771" watchObservedRunningTime="2026-08-04 09:44:45.159509873 +0000 UTC m=+2216.417466928"
Aug 04 09:44:46.148044 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:46.147991    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk" event={"ID":"157bd973-c8d4-4199-b27f-c5e3e73d4698","Type":"ContainerStarted","Data":"bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2"}
Aug 04 09:44:46.148459 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:46.148077    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:44:46.165039 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:46.164991    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk" podStartSLOduration=2.164977306 podStartE2EDuration="2.164977306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:44:46.164383662 +0000 UTC m=+2217.422340717" watchObservedRunningTime="2026-08-04 09:44:46.164977306 +0000 UTC m=+2217.422934359"
Aug 04 09:44:49.437606 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:49.437569    2558 scope.go:117] "RemoveContainer" containerID="8febc0b6594409151e342e679d531731562c17c739a7b8439c4e9b3b7076a122"
Aug 04 09:44:50.542587 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:50.542551    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:44:56.149894 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:56.149863    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:44:57.156494 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:44:57.156461    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:45:00.145642 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:00.145607    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"]
Aug 04 09:45:00.148928 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:00.148910    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"
Aug 04 09:45:00.151527 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:00.151508    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\""
Aug 04 09:45:00.158279 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:00.158253    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"]
Aug 04 09:45:00.159901 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:00.159867    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"
Aug 04 09:45:00.303039 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:00.303002    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"]
Aug 04 09:45:00.309784 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:45:00.309759    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df3f03d_c47b_42ed_8606_f2faaa6eaa82.slice/crio-f8f3bf991aac5e596a746dede2bf874d72f53d7ca7fc45fd8ac57759ace3da48 WatchSource:0}: Error finding container f8f3bf991aac5e596a746dede2bf874d72f53d7ca7fc45fd8ac57759ace3da48: Status 404 returned error can't find the container with id f8f3bf991aac5e596a746dede2bf874d72f53d7ca7fc45fd8ac57759ace3da48
Aug 04 09:45:01.229327 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:01.229295    2558 generic.go:356] "Generic (PLEG): container finished" podID="4df3f03d-c47b-42ed-8606-f2faaa6eaa82" containerID="50a9a65a61f03d39787d8ff4830512c5b60de76c8cc8d292bf845243754496cd" exitCode=0
Aug 04 09:45:01.229679 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:01.229366    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2" event={"ID":"4df3f03d-c47b-42ed-8606-f2faaa6eaa82","Type":"ContainerDied","Data":"50a9a65a61f03d39787d8ff4830512c5b60de76c8cc8d292bf845243754496cd"}
Aug 04 09:45:01.229679 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:01.229439    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2" event={"ID":"4df3f03d-c47b-42ed-8606-f2faaa6eaa82","Type":"ContainerStarted","Data":"f8f3bf991aac5e596a746dede2bf874d72f53d7ca7fc45fd8ac57759ace3da48"}
Aug 04 09:45:02.373467 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:02.373442    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"
Aug 04 09:45:03.239568 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:03.239540    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"
Aug 04 09:45:03.239754 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:03.239533    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2" event={"ID":"4df3f03d-c47b-42ed-8606-f2faaa6eaa82","Type":"ContainerDied","Data":"f8f3bf991aac5e596a746dede2bf874d72f53d7ca7fc45fd8ac57759ace3da48"}
Aug 04 09:45:03.239754 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:03.239664    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8f3bf991aac5e596a746dede2bf874d72f53d7ca7fc45fd8ac57759ace3da48"
Aug 04 09:45:23.442664 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.442584    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"]
Aug 04 09:45:23.443148 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.443008    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4df3f03d-c47b-42ed-8606-f2faaa6eaa82" containerName="cleanup"
Aug 04 09:45:23.443148 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.443022    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4df3f03d-c47b-42ed-8606-f2faaa6eaa82" containerName="cleanup"
Aug 04 09:45:23.443148 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.443087    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4df3f03d-c47b-42ed-8606-f2faaa6eaa82" containerName="cleanup"
Aug 04 09:45:23.445947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.445932    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.448787 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.448767    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-18d97d1d\"/\"default-dockercfg-6gczp\""
Aug 04 09:45:23.449626 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.449609    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-18d97d1d\"/\"e2e-lifecycle-model-18d97d1d-kserve-self-signed-certs\""
Aug 04 09:45:23.449683 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.449669    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-18d97d1d\"/\"kube-root-ca.crt\""
Aug 04 09:45:23.449722 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.449666    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-18d97d1d\"/\"openshift-service-ca.crt\""
Aug 04 09:45:23.457828 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.457804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"]
Aug 04 09:45:23.492126 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.492098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq2gr\" (UniqueName: \"kubernetes.io/projected/145172b9-b653-4f08-b5d6-debab61163b7-kube-api-access-pq2gr\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.492254 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.492173    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-model-cache\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.492254 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.492248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145172b9-b653-4f08-b5d6-debab61163b7-tls-certs\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.492362 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.492337    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-tmp-dir\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.492471 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.492449    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-home\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.492606 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.492514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-dshm\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.593716 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.593678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-dshm\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.593716 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.593720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq2gr\" (UniqueName: \"kubernetes.io/projected/145172b9-b653-4f08-b5d6-debab61163b7-kube-api-access-pq2gr\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.593974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.593774    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-model-cache\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.593974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.593822    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145172b9-b653-4f08-b5d6-debab61163b7-tls-certs\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.593974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.593948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-tmp-dir\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.594141 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.594045    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-home\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.594195 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.594174    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-model-cache\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.594262 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.594247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-tmp-dir\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.594335 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.594321    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-home\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.596227 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.596203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-dshm\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.596384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.596369    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145172b9-b653-4f08-b5d6-debab61163b7-tls-certs\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.603342 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.603322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq2gr\" (UniqueName: \"kubernetes.io/projected/145172b9-b653-4f08-b5d6-debab61163b7-kube-api-access-pq2gr\") pod \"e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") " pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.755623 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.755546    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:23.890493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:23.890469    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"]
Aug 04 09:45:23.892462 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:45:23.892433    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod145172b9_b653_4f08_b5d6_debab61163b7.slice/crio-7f69820d24685d46a6e7df12b1f01d5bb0e32202adc84db4d98511858fc790b7 WatchSource:0}: Error finding container 7f69820d24685d46a6e7df12b1f01d5bb0e32202adc84db4d98511858fc790b7: Status 404 returned error can't find the container with id 7f69820d24685d46a6e7df12b1f01d5bb0e32202adc84db4d98511858fc790b7
Aug 04 09:45:24.344684 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:24.344648    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc" event={"ID":"145172b9-b653-4f08-b5d6-debab61163b7","Type":"ContainerStarted","Data":"53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97"}
Aug 04 09:45:24.344684 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:24.344694    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc" event={"ID":"145172b9-b653-4f08-b5d6-debab61163b7","Type":"ContainerStarted","Data":"7f69820d24685d46a6e7df12b1f01d5bb0e32202adc84db4d98511858fc790b7"}
Aug 04 09:45:24.362850 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:24.362805    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc" podStartSLOduration=1.362788795 podStartE2EDuration="1.362788795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:24.361274925 +0000 UTC m=+2255.619231980" watchObservedRunningTime="2026-08-04 09:45:24.362788795 +0000 UTC m=+2255.620745849"
Aug 04 09:45:26.037547 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:26.037509    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:45:33.755808 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:33.755772    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:33.756213 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:33.755822    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:33.768250 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:33.768224    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:34.403096 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:34.403063    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:45:41.348322 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:41.348284    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:45:49.455380 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:49.455348    2558 scope.go:117] "RemoveContainer" containerID="388b2fb5c1658e3f467781c41bb4cdca2bae5e3316be10debed036bf9512ef91"
Aug 04 09:45:49.464066 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:45:49.464046    2558 scope.go:117] "RemoveContainer" containerID="6c2f150ae657a3e39e0c6cd877586d2d90e99f03fb0caa1601df761d6bc895b8"
Aug 04 09:46:05.235799 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:05.235760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:46:07.565901 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.565866    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"]
Aug 04 09:46:07.568868 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.568835    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-18d97d1d-29763945-pdpw2"]
Aug 04 09:46:07.613505 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.613474    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"]
Aug 04 09:46:07.613686 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.613665    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk" podUID="157bd973-c8d4-4199-b27f-c5e3e73d4698" containerName="payload-processing" containerID="cri-o://bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2" gracePeriod=30
Aug 04 09:46:07.633363 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.633337    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"]
Aug 04 09:46:07.633535 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.633515    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx" podUID="e0edc384-a78f-4cbe-ae20-5c113d7b5244" containerName="payload-pre-processing" containerID="cri-o://e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445" gracePeriod=30
Aug 04 09:46:07.914730 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.914705    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:46:07.917577 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.917541    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:46:07.995935 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.995903    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkblb\" (UniqueName: \"kubernetes.io/projected/e0edc384-a78f-4cbe-ae20-5c113d7b5244-kube-api-access-mkblb\") pod \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") "
Aug 04 09:46:07.996095 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.995954    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0edc384-a78f-4cbe-ae20-5c113d7b5244-plugins-config-volume\") pod \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\" (UID: \"e0edc384-a78f-4cbe-ae20-5c113d7b5244\") "
Aug 04 09:46:07.996095 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.995984    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/157bd973-c8d4-4199-b27f-c5e3e73d4698-plugins-config-volume\") pod \"157bd973-c8d4-4199-b27f-c5e3e73d4698\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") "
Aug 04 09:46:07.996213 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.996094    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skb8h\" (UniqueName: \"kubernetes.io/projected/157bd973-c8d4-4199-b27f-c5e3e73d4698-kube-api-access-skb8h\") pod \"157bd973-c8d4-4199-b27f-c5e3e73d4698\" (UID: \"157bd973-c8d4-4199-b27f-c5e3e73d4698\") "
Aug 04 09:46:07.996367 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.996347    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0edc384-a78f-4cbe-ae20-5c113d7b5244-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0edc384-a78f-4cbe-ae20-5c113d7b5244" (UID: "e0edc384-a78f-4cbe-ae20-5c113d7b5244"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:46:07.996428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.996382    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/157bd973-c8d4-4199-b27f-c5e3e73d4698-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "157bd973-c8d4-4199-b27f-c5e3e73d4698" (UID: "157bd973-c8d4-4199-b27f-c5e3e73d4698"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:46:07.998380 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.998357    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0edc384-a78f-4cbe-ae20-5c113d7b5244-kube-api-access-mkblb" (OuterVolumeSpecName: "kube-api-access-mkblb") pod "e0edc384-a78f-4cbe-ae20-5c113d7b5244" (UID: "e0edc384-a78f-4cbe-ae20-5c113d7b5244"). InnerVolumeSpecName "kube-api-access-mkblb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:46:07.998480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:07.998387    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/157bd973-c8d4-4199-b27f-c5e3e73d4698-kube-api-access-skb8h" (OuterVolumeSpecName: "kube-api-access-skb8h") pod "157bd973-c8d4-4199-b27f-c5e3e73d4698" (UID: "157bd973-c8d4-4199-b27f-c5e3e73d4698"). InnerVolumeSpecName "kube-api-access-skb8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:46:08.097161 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.097087    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skb8h\" (UniqueName: \"kubernetes.io/projected/157bd973-c8d4-4199-b27f-c5e3e73d4698-kube-api-access-skb8h\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:08.097161 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.097113    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkblb\" (UniqueName: \"kubernetes.io/projected/e0edc384-a78f-4cbe-ae20-5c113d7b5244-kube-api-access-mkblb\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:08.097161 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.097122    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0edc384-a78f-4cbe-ae20-5c113d7b5244-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:08.097161 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.097130    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/157bd973-c8d4-4199-b27f-c5e3e73d4698-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:08.550219 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.550132    2558 generic.go:356] "Generic (PLEG): container finished" podID="e0edc384-a78f-4cbe-ae20-5c113d7b5244" containerID="e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445" exitCode=0
Aug 04 09:46:08.550219 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.550177    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"
Aug 04 09:46:08.550420 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.550217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx" event={"ID":"e0edc384-a78f-4cbe-ae20-5c113d7b5244","Type":"ContainerDied","Data":"e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445"}
Aug 04 09:46:08.550420 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.550281    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx" event={"ID":"e0edc384-a78f-4cbe-ae20-5c113d7b5244","Type":"ContainerDied","Data":"d7e49c6788e9d8a5cf91207646e01504302e3479f486cab91f48b6f7f3957ce7"}
Aug 04 09:46:08.550420 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.550320    2558 scope.go:117] "RemoveContainer" containerID="e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445"
Aug 04 09:46:08.551537 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.551516    2558 generic.go:356] "Generic (PLEG): container finished" podID="157bd973-c8d4-4199-b27f-c5e3e73d4698" containerID="bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2" exitCode=0
Aug 04 09:46:08.551646 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.551565    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"
Aug 04 09:46:08.551646 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.551575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk" event={"ID":"157bd973-c8d4-4199-b27f-c5e3e73d4698","Type":"ContainerDied","Data":"bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2"}
Aug 04 09:46:08.551646 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.551630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk" event={"ID":"157bd973-c8d4-4199-b27f-c5e3e73d4698","Type":"ContainerDied","Data":"384e5c78bc14021c87275f6aafb0854f75e3aede681e166d71f91b73c14344d2"}
Aug 04 09:46:08.559723 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.559703    2558 scope.go:117] "RemoveContainer" containerID="e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445"
Aug 04 09:46:08.560000 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:46:08.559978    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445\": container with ID starting with e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445 not found: ID does not exist" containerID="e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445"
Aug 04 09:46:08.560065 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.560011    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445"} err="failed to get container status \"e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445\": rpc error: code = NotFound desc = could not find container \"e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445\": container with ID starting with e267d8ac5d96acbbd0151057627b04d1b6f187be8f477143bce792746f992445 not found: ID does not exist"
Aug 04 09:46:08.560065 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.560029    2558 scope.go:117] "RemoveContainer" containerID="bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2"
Aug 04 09:46:08.567718 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.567701    2558 scope.go:117] "RemoveContainer" containerID="bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2"
Aug 04 09:46:08.568013 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:46:08.567952    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2\": container with ID starting with bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2 not found: ID does not exist" containerID="bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2"
Aug 04 09:46:08.568013 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.567973    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2"} err="failed to get container status \"bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2\": rpc error: code = NotFound desc = could not find container \"bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2\": container with ID starting with bd7fa7657ee3bceb0bc09fce9183cb30dbee8dd9aa08fb6664bff7b999bb14d2 not found: ID does not exist"
Aug 04 09:46:08.575854 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.575831    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"]
Aug 04 09:46:08.581460 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.581440    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-18d97d1d-68b5b49766-xlnhk"]
Aug 04 09:46:08.592973 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.592954    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"]
Aug 04 09:46:08.597532 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:08.597512    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-18d97d1d-6fdb67895b-7ltdx"]
Aug 04 09:46:09.296553 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:09.296520    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="157bd973-c8d4-4199-b27f-c5e3e73d4698" path="/var/lib/kubelet/pods/157bd973-c8d4-4199-b27f-c5e3e73d4698/volumes"
Aug 04 09:46:09.296819 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:09.296806    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4df3f03d-c47b-42ed-8606-f2faaa6eaa82" path="/var/lib/kubelet/pods/4df3f03d-c47b-42ed-8606-f2faaa6eaa82/volumes"
Aug 04 09:46:09.297085 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:09.297074    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0edc384-a78f-4cbe-ae20-5c113d7b5244" path="/var/lib/kubelet/pods/e0edc384-a78f-4cbe-ae20-5c113d7b5244/volumes"
Aug 04 09:46:11.139728 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:11.139692    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:46:16.182072 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:16.182033    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"]
Aug 04 09:46:31.202027 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.201960    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc" podUID="145172b9-b653-4f08-b5d6-debab61163b7" containerName="main" containerID="cri-o://53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97" gracePeriod=45
Aug 04 09:46:31.456705 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.456638    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:46:31.601794 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.601758    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-model-cache\") pod \"145172b9-b653-4f08-b5d6-debab61163b7\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") "
Aug 04 09:46:31.601982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.601819    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-home\") pod \"145172b9-b653-4f08-b5d6-debab61163b7\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") "
Aug 04 09:46:31.601982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.601841    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-dshm\") pod \"145172b9-b653-4f08-b5d6-debab61163b7\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") "
Aug 04 09:46:31.601982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.601879    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145172b9-b653-4f08-b5d6-debab61163b7-tls-certs\") pod \"145172b9-b653-4f08-b5d6-debab61163b7\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") "
Aug 04 09:46:31.601982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.601893    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-tmp-dir\") pod \"145172b9-b653-4f08-b5d6-debab61163b7\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") "
Aug 04 09:46:31.601982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.601941    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq2gr\" (UniqueName: \"kubernetes.io/projected/145172b9-b653-4f08-b5d6-debab61163b7-kube-api-access-pq2gr\") pod \"145172b9-b653-4f08-b5d6-debab61163b7\" (UID: \"145172b9-b653-4f08-b5d6-debab61163b7\") "
Aug 04 09:46:31.602227 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.602073    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-home" (OuterVolumeSpecName: "home") pod "145172b9-b653-4f08-b5d6-debab61163b7" (UID: "145172b9-b653-4f08-b5d6-debab61163b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:46:31.602227 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.602085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "145172b9-b653-4f08-b5d6-debab61163b7" (UID: "145172b9-b653-4f08-b5d6-debab61163b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:46:31.602227 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.602180    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "145172b9-b653-4f08-b5d6-debab61163b7" (UID: "145172b9-b653-4f08-b5d6-debab61163b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:46:31.602387 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.602285    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-home\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:31.602387 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.602314    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-tmp-dir\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:31.602387 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.602324    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-model-cache\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:31.604152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.604128    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-dshm" (OuterVolumeSpecName: "dshm") pod "145172b9-b653-4f08-b5d6-debab61163b7" (UID: "145172b9-b653-4f08-b5d6-debab61163b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:46:31.604248 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.604199    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/145172b9-b653-4f08-b5d6-debab61163b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "145172b9-b653-4f08-b5d6-debab61163b7" (UID: "145172b9-b653-4f08-b5d6-debab61163b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 09:46:31.604248 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.604226    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/145172b9-b653-4f08-b5d6-debab61163b7-kube-api-access-pq2gr" (OuterVolumeSpecName: "kube-api-access-pq2gr") pod "145172b9-b653-4f08-b5d6-debab61163b7" (UID: "145172b9-b653-4f08-b5d6-debab61163b7"). InnerVolumeSpecName "kube-api-access-pq2gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:46:31.666964 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.666936    2558 generic.go:356] "Generic (PLEG): container finished" podID="145172b9-b653-4f08-b5d6-debab61163b7" containerID="53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97" exitCode=0
Aug 04 09:46:31.667089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.666980    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"
Aug 04 09:46:31.667089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.667023    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc" event={"ID":"145172b9-b653-4f08-b5d6-debab61163b7","Type":"ContainerDied","Data":"53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97"}
Aug 04 09:46:31.667089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.667065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc" event={"ID":"145172b9-b653-4f08-b5d6-debab61163b7","Type":"ContainerDied","Data":"7f69820d24685d46a6e7df12b1f01d5bb0e32202adc84db4d98511858fc790b7"}
Aug 04 09:46:31.667089 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.667082    2558 scope.go:117] "RemoveContainer" containerID="53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97"
Aug 04 09:46:31.676784 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.676767    2558 scope.go:117] "RemoveContainer" containerID="53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97"
Aug 04 09:46:31.677072 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:46:31.677045    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97\": container with ID starting with 53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97 not found: ID does not exist" containerID="53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97"
Aug 04 09:46:31.677179 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.677079    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97"} err="failed to get container status \"53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97\": rpc error: code = NotFound desc = could not find container \"53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97\": container with ID starting with 53f1210609605a500bc3ee8348a9e2a68fac65beb836cc21507167068901cf97 not found: ID does not exist"
Aug 04 09:46:31.690488 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.690463    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"]
Aug 04 09:46:31.696889 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.696869    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-18d97d1d/e2e-lifecycle-model-18d97d1d-kserve-597f7bb88c-f4phc"]
Aug 04 09:46:31.703115 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.703094    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145172b9-b653-4f08-b5d6-debab61163b7-dshm\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:31.703188 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.703120    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145172b9-b653-4f08-b5d6-debab61163b7-tls-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:31.703188 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:31.703130    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq2gr\" (UniqueName: \"kubernetes.io/projected/145172b9-b653-4f08-b5d6-debab61163b7-kube-api-access-pq2gr\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:46:33.295778 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:46:33.295725    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="145172b9-b653-4f08-b5d6-debab61163b7" path="/var/lib/kubelet/pods/145172b9-b653-4f08-b5d6-debab61163b7/volumes"
Aug 04 09:47:02.322901 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:47:02.322863    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:47:14.350264 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:47:14.350233    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:47:20.953767 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:47:20.953721    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:47:28.979316 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:47:28.979268    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:48:13.398259 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398176    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"]
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398550    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="157bd973-c8d4-4199-b27f-c5e3e73d4698" containerName="payload-processing"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398564    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="157bd973-c8d4-4199-b27f-c5e3e73d4698" containerName="payload-processing"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398576    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="145172b9-b653-4f08-b5d6-debab61163b7" containerName="main"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398582    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="145172b9-b653-4f08-b5d6-debab61163b7" containerName="main"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398590    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0edc384-a78f-4cbe-ae20-5c113d7b5244" containerName="payload-pre-processing"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398595    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0edc384-a78f-4cbe-ae20-5c113d7b5244" containerName="payload-pre-processing"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398649    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="157bd973-c8d4-4199-b27f-c5e3e73d4698" containerName="payload-processing"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398656    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="145172b9-b653-4f08-b5d6-debab61163b7" containerName="main"
Aug 04 09:48:13.398769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.398675    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0edc384-a78f-4cbe-ae20-5c113d7b5244" containerName="payload-pre-processing"
Aug 04 09:48:13.401691 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.401675    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.404575 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.404556    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-4d301b\""
Aug 04 09:48:13.413700 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.413676    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"]
Aug 04 09:48:13.559365 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.559327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da15f22f-88d2-4ec2-8cfd-5aae29273021-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.559553 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.559404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cdjp\" (UniqueName: \"kubernetes.io/projected/da15f22f-88d2-4ec2-8cfd-5aae29273021-kube-api-access-9cdjp\") pod \"payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.659963 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.659880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da15f22f-88d2-4ec2-8cfd-5aae29273021-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.659963 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.659944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cdjp\" (UniqueName: \"kubernetes.io/projected/da15f22f-88d2-4ec2-8cfd-5aae29273021-kube-api-access-9cdjp\") pod \"payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.660610 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.660587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da15f22f-88d2-4ec2-8cfd-5aae29273021-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.670048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.670024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cdjp\" (UniqueName: \"kubernetes.io/projected/da15f22f-88d2-4ec2-8cfd-5aae29273021-kube-api-access-9cdjp\") pod \"payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.711009 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.710974    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:13.857832 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:13.857807    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"]
Aug 04 09:48:13.859948 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:48:13.859922    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda15f22f_88d2_4ec2_8cfd_5aae29273021.slice/crio-99fe12bd34f245a9b2528e4ad76a595c7b4c693c67da6a9c7fa5469a3849fb0f WatchSource:0}: Error finding container 99fe12bd34f245a9b2528e4ad76a595c7b4c693c67da6a9c7fa5469a3849fb0f: Status 404 returned error can't find the container with id 99fe12bd34f245a9b2528e4ad76a595c7b4c693c67da6a9c7fa5469a3849fb0f
Aug 04 09:48:14.161928 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:14.161889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb" event={"ID":"da15f22f-88d2-4ec2-8cfd-5aae29273021","Type":"ContainerStarted","Data":"21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd"}
Aug 04 09:48:14.162101 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:14.161944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb" event={"ID":"da15f22f-88d2-4ec2-8cfd-5aae29273021","Type":"ContainerStarted","Data":"99fe12bd34f245a9b2528e4ad76a595c7b4c693c67da6a9c7fa5469a3849fb0f"}
Aug 04 09:48:14.162101 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:14.162037    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:14.188080 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:14.187971    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb" podStartSLOduration=1.187956079 podStartE2EDuration="1.187956079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:14.186727225 +0000 UTC m=+2425.444684283" watchObservedRunningTime="2026-08-04 09:48:14.187956079 +0000 UTC m=+2425.445913132"
Aug 04 09:48:15.840868 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:15.840834    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:48:25.169182 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:25.169151    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:48:36.938132 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:36.938098    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"]
Aug 04 09:48:36.941209 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:36.941186    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:36.943865 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:36.943843    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-406b0d-openshift-default-dockercfg-mwgc9\""
Aug 04 09:48:36.958569 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:36.958545    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"]
Aug 04 09:48:37.057402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-socket\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057410    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-certs\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-credential-socket\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057531    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-envoy\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057560    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-data\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057774 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-podinfo\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057774 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057671    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452d27b3-a539-4efc-94f8-6df6a6c1651c-istiod-ca-cert\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057774 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057699    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqqr2\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-kube-api-access-hqqr2\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.057892 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.057776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-token\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.158705 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.158667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-certs\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.158898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.158754    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-credential-socket\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.158898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.158784    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-envoy\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.158898 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.158811    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-data\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159090 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.158955    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-podinfo\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159090 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159028    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452d27b3-a539-4efc-94f8-6df6a6c1651c-istiod-ca-cert\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159090 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159053    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqqr2\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-kube-api-access-hqqr2\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159242 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159101    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-token\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159242 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-socket\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159242 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-credential-socket\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159445 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159350    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-certs\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159445 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159420    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-data\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159532 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159483    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-socket\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.159938 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.159914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452d27b3-a539-4efc-94f8-6df6a6c1651c-istiod-ca-cert\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.161424 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.161405    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-envoy\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.161640 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.161619    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-podinfo\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.172569 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.172545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-token\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.172711 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.172690    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqqr2\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-kube-api-access-hqqr2\") pod \"e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") " pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.251329 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.251262    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:37.402048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.402022    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"]
Aug 04 09:48:37.404080 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:48:37.404054    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452d27b3_a539_4efc_94f8_6df6a6c1651c.slice/crio-02265eb61b56225397d278c2f5b0cb9014ac73bb800ff882deec767b99e44110 WatchSource:0}: Error finding container 02265eb61b56225397d278c2f5b0cb9014ac73bb800ff882deec767b99e44110: Status 404 returned error can't find the container with id 02265eb61b56225397d278c2f5b0cb9014ac73bb800ff882deec767b99e44110
Aug 04 09:48:37.406273 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.406241    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:48:37.406361 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.406303    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:48:37.406361 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:37.406338    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:48:38.285008 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:38.284972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" event={"ID":"452d27b3-a539-4efc-94f8-6df6a6c1651c","Type":"ContainerStarted","Data":"d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73"}
Aug 04 09:48:38.285493 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:38.285033    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" event={"ID":"452d27b3-a539-4efc-94f8-6df6a6c1651c","Type":"ContainerStarted","Data":"02265eb61b56225397d278c2f5b0cb9014ac73bb800ff882deec767b99e44110"}
Aug 04 09:48:38.308567 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:38.308521    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podStartSLOduration=2.308508378 podStartE2EDuration="2.308508378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:38.306969162 +0000 UTC m=+2449.564926251" watchObservedRunningTime="2026-08-04 09:48:38.308508378 +0000 UTC m=+2449.566465433"
Aug 04 09:48:39.251801 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:39.251772    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:39.256848 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:39.256825    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:39.296019 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:39.295992    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:39.296363 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:39.296049    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:42.357965 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.357935    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"]
Aug 04 09:48:42.361753 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.361719    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.366526 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.366509    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-406b0d\""
Aug 04 09:48:42.367270 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.367249    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-406b0d-dockercfg-785tf\""
Aug 04 09:48:42.381061 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.381041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"]
Aug 04 09:48:42.413000 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.412976    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shmlh\" (UniqueName: \"kubernetes.io/projected/8e74da58-337e-400c-bee0-f7b5bc597904-kube-api-access-shmlh\") pod \"payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") " pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.413127 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.413043    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e74da58-337e-400c-bee0-f7b5bc597904-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") " pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.514327 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.514300    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e74da58-337e-400c-bee0-f7b5bc597904-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") " pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.514476 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.514355    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shmlh\" (UniqueName: \"kubernetes.io/projected/8e74da58-337e-400c-bee0-f7b5bc597904-kube-api-access-shmlh\") pod \"payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") " pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.515009 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.514992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e74da58-337e-400c-bee0-f7b5bc597904-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") " pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.527283 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.527248    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shmlh\" (UniqueName: \"kubernetes.io/projected/8e74da58-337e-400c-bee0-f7b5bc597904-kube-api-access-shmlh\") pod \"payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") " pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.672638 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.672545    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:42.808290 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:42.808253    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"]
Aug 04 09:48:42.811551 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:48:42.811516    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e74da58_337e_400c_bee0_f7b5bc597904.slice/crio-80657b488b5e19eaf014d07f238867a69ba192bf6492a8ad38b85024a5f7ca65 WatchSource:0}: Error finding container 80657b488b5e19eaf014d07f238867a69ba192bf6492a8ad38b85024a5f7ca65: Status 404 returned error can't find the container with id 80657b488b5e19eaf014d07f238867a69ba192bf6492a8ad38b85024a5f7ca65
Aug 04 09:48:43.316604 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:43.316574    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2" event={"ID":"8e74da58-337e-400c-bee0-f7b5bc597904","Type":"ContainerStarted","Data":"c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34"}
Aug 04 09:48:43.316836 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:43.316619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2" event={"ID":"8e74da58-337e-400c-bee0-f7b5bc597904","Type":"ContainerStarted","Data":"80657b488b5e19eaf014d07f238867a69ba192bf6492a8ad38b85024a5f7ca65"}
Aug 04 09:48:43.316836 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:43.316664    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:43.338252 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:43.338194    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2" podStartSLOduration=1.338175971 podStartE2EDuration="1.338175971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:43.337426144 +0000 UTC m=+2454.595383199" watchObservedRunningTime="2026-08-04 09:48:43.338175971 +0000 UTC m=+2454.596133031"
Aug 04 09:48:44.951714 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:44.951676    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:48:49.515408 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:49.515377    2558 scope.go:117] "RemoveContainer" containerID="574b5af8e9663ed89e39adc5137e0ba70ca67fdf8cf3caf6d8a93339f9f486b7"
Aug 04 09:48:54.323443 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:54.323406    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:48:54.789287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:54.788642    2558 patch_prober.go:28] interesting pod/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:37546->10.133.0.52:15021: read: connection reset by peer" start-of-body=
Aug 04 09:48:54.789287 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:54.788722    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:37546->10.133.0.52:15021: read: connection reset by peer"
Aug 04 09:48:55.396292 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:55.396258    2558 generic.go:356] "Generic (PLEG): container finished" podID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerID="d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73" exitCode=0
Aug 04 09:48:55.396776 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:55.396304    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" event={"ID":"452d27b3-a539-4efc-94f8-6df6a6c1651c","Type":"ContainerDied","Data":"d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73"}
Aug 04 09:48:55.396870 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:55.396774    2558 scope.go:117] "RemoveContainer" containerID="d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73"
Aug 04 09:48:55.397756 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:55.397718    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:48:55.397820 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:55.397799    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:48:55.397856 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:55.397828    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:48:56.402810 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:56.402777    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" event={"ID":"452d27b3-a539-4efc-94f8-6df6a6c1651c","Type":"ContainerStarted","Data":"6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58"}
Aug 04 09:48:57.252321 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:57.252281    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:48:57.252518 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:57.252495    2558 patch_prober.go:28] interesting pod/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Aug 04 09:48:57.252575 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:57.252555    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Aug 04 09:48:58.251982 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:58.251942    2558 patch_prober.go:28] interesting pod/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Aug 04 09:48:58.252401 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:58.252013    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Aug 04 09:48:59.252030 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:59.251992    2558 patch_prober.go:28] interesting pod/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Aug 04 09:48:59.252416 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:48:59.252063    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Aug 04 09:49:00.252512 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:00.252468    2558 patch_prober.go:28] interesting pod/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Aug 04 09:49:00.252926 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:00.252529    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Aug 04 09:49:01.251676 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:01.251636    2558 patch_prober.go:28] interesting pod/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Aug 04 09:49:01.251887 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:01.251695    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Aug 04 09:49:02.255116 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:02.255088    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:49:02.255495 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:02.255356    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:49:02.256070 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:02.256053    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:49:26.662202 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:26.662165    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:49:33.854314 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:33.854276    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:49:50.854096 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:49:50.854061    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:50:02.164962 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:02.164923    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:50:31.599012 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.598976    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"]
Aug 04 09:50:31.603162 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.603133    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"
Aug 04 09:50:31.607400 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.607379    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\""
Aug 04 09:50:31.638286 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.638245    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"]
Aug 04 09:50:31.651407 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.651378    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:50:31.659965 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.659927    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e311bc79-87cc-444d-816c-5391109cbf23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d301b-979jk\" (UID: \"e311bc79-87cc-444d-816c-5391109cbf23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"
Aug 04 09:50:31.760916 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.760880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e311bc79-87cc-444d-816c-5391109cbf23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d301b-979jk\" (UID: \"e311bc79-87cc-444d-816c-5391109cbf23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"
Aug 04 09:50:31.761631 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.761600    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e311bc79-87cc-444d-816c-5391109cbf23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d301b-979jk\" (UID: \"e311bc79-87cc-444d-816c-5391109cbf23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"
Aug 04 09:50:31.914431 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:31.914344    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"
Aug 04 09:50:32.064454 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:32.064428    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"]
Aug 04 09:50:32.067114 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:50:32.067088    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode311bc79_87cc_444d_816c_5391109cbf23.slice/crio-10f3f22c058b1acd44881565be0ae6e5145cc76459fbff2a4472e48271f978c1 WatchSource:0}: Error finding container 10f3f22c058b1acd44881565be0ae6e5145cc76459fbff2a4472e48271f978c1: Status 404 returned error can't find the container with id 10f3f22c058b1acd44881565be0ae6e5145cc76459fbff2a4472e48271f978c1
Aug 04 09:50:32.852230 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:32.852194    2558 generic.go:356] "Generic (PLEG): container finished" podID="e311bc79-87cc-444d-816c-5391109cbf23" containerID="e309ec8720998aace51fd78ec630cb020aa49257969313dbf17adba2b75d0d90" exitCode=0
Aug 04 09:50:32.852591 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:32.852318    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk" event={"ID":"e311bc79-87cc-444d-816c-5391109cbf23","Type":"ContainerDied","Data":"e309ec8720998aace51fd78ec630cb020aa49257969313dbf17adba2b75d0d90"}
Aug 04 09:50:32.852591 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:32.852388    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk" event={"ID":"e311bc79-87cc-444d-816c-5391109cbf23","Type":"ContainerStarted","Data":"10f3f22c058b1acd44881565be0ae6e5145cc76459fbff2a4472e48271f978c1"}
Aug 04 09:50:33.992834 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:33.992810    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"
Aug 04 09:50:34.082953 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:34.082922    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e311bc79-87cc-444d-816c-5391109cbf23-maas-api-service-ca\") pod \"e311bc79-87cc-444d-816c-5391109cbf23\" (UID: \"e311bc79-87cc-444d-816c-5391109cbf23\") "
Aug 04 09:50:34.083244 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:34.083223    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e311bc79-87cc-444d-816c-5391109cbf23-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e311bc79-87cc-444d-816c-5391109cbf23" (UID: "e311bc79-87cc-444d-816c-5391109cbf23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:50:34.183723 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:34.183651    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e311bc79-87cc-444d-816c-5391109cbf23-maas-api-service-ca\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:50:34.862170 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:34.862138    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"
Aug 04 09:50:34.862337 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:34.862176    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk" event={"ID":"e311bc79-87cc-444d-816c-5391109cbf23","Type":"ContainerDied","Data":"10f3f22c058b1acd44881565be0ae6e5145cc76459fbff2a4472e48271f978c1"}
Aug 04 09:50:34.862337 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:34.862223    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10f3f22c058b1acd44881565be0ae6e5145cc76459fbff2a4472e48271f978c1"
Aug 04 09:50:36.012488 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.012455    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"]
Aug 04 09:50:36.012899 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.012653    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb" podUID="da15f22f-88d2-4ec2-8cfd-5aae29273021" containerName="payload-pre-processing" containerID="cri-o://21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd" gracePeriod=30
Aug 04 09:50:36.280788 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.280754    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:50:36.301914 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.301888    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"]
Aug 04 09:50:36.308705 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.308678    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d301b-979jk"]
Aug 04 09:50:36.400349 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.400311    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da15f22f-88d2-4ec2-8cfd-5aae29273021-plugins-config-volume\") pod \"da15f22f-88d2-4ec2-8cfd-5aae29273021\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") "
Aug 04 09:50:36.400510 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.400432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cdjp\" (UniqueName: \"kubernetes.io/projected/da15f22f-88d2-4ec2-8cfd-5aae29273021-kube-api-access-9cdjp\") pod \"da15f22f-88d2-4ec2-8cfd-5aae29273021\" (UID: \"da15f22f-88d2-4ec2-8cfd-5aae29273021\") "
Aug 04 09:50:36.400665 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.400644    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da15f22f-88d2-4ec2-8cfd-5aae29273021-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da15f22f-88d2-4ec2-8cfd-5aae29273021" (UID: "da15f22f-88d2-4ec2-8cfd-5aae29273021"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:50:36.402418 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.402396    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da15f22f-88d2-4ec2-8cfd-5aae29273021-kube-api-access-9cdjp" (OuterVolumeSpecName: "kube-api-access-9cdjp") pod "da15f22f-88d2-4ec2-8cfd-5aae29273021" (UID: "da15f22f-88d2-4ec2-8cfd-5aae29273021"). InnerVolumeSpecName "kube-api-access-9cdjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:50:36.501216 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.501185    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cdjp\" (UniqueName: \"kubernetes.io/projected/da15f22f-88d2-4ec2-8cfd-5aae29273021-kube-api-access-9cdjp\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:50:36.501216 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.501212    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da15f22f-88d2-4ec2-8cfd-5aae29273021-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:50:36.874577 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.874541    2558 generic.go:356] "Generic (PLEG): container finished" podID="da15f22f-88d2-4ec2-8cfd-5aae29273021" containerID="21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd" exitCode=0
Aug 04 09:50:36.874779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.874584    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"
Aug 04 09:50:36.874779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.874627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb" event={"ID":"da15f22f-88d2-4ec2-8cfd-5aae29273021","Type":"ContainerDied","Data":"21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd"}
Aug 04 09:50:36.874779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.874678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb" event={"ID":"da15f22f-88d2-4ec2-8cfd-5aae29273021","Type":"ContainerDied","Data":"99fe12bd34f245a9b2528e4ad76a595c7b4c693c67da6a9c7fa5469a3849fb0f"}
Aug 04 09:50:36.874779 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.874713    2558 scope.go:117] "RemoveContainer" containerID="21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd"
Aug 04 09:50:36.884094 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.884077    2558 scope.go:117] "RemoveContainer" containerID="21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd"
Aug 04 09:50:36.884374 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:50:36.884356    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd\": container with ID starting with 21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd not found: ID does not exist" containerID="21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd"
Aug 04 09:50:36.884423 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.884386    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd"} err="failed to get container status \"21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd\": rpc error: code = NotFound desc = could not find container \"21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd\": container with ID starting with 21410cd56824b4c48e0e0c3a9bebd1dfa73c08ec900e1495990ecc624e0405cd not found: ID does not exist"
Aug 04 09:50:36.902402 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.902378    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"]
Aug 04 09:50:36.909821 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:36.909798    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4d301b-685dc55896-xbcwb"]
Aug 04 09:50:37.295501 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:37.295424    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da15f22f-88d2-4ec2-8cfd-5aae29273021" path="/var/lib/kubelet/pods/da15f22f-88d2-4ec2-8cfd-5aae29273021/volumes"
Aug 04 09:50:37.295959 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:37.295730    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e311bc79-87cc-444d-816c-5391109cbf23" path="/var/lib/kubelet/pods/e311bc79-87cc-444d-816c-5391109cbf23/volumes"
Aug 04 09:50:38.048706 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:38.048670    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:50:58.992387 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:58.992349    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"]
Aug 04 09:50:58.992852 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:58.992604    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2" podUID="8e74da58-337e-400c-bee0-f7b5bc597904" containerName="payload-processing" containerID="cri-o://c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34" gracePeriod=30
Aug 04 09:50:59.252715 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.252641    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:50:59.401649 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.401613    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e74da58-337e-400c-bee0-f7b5bc597904-plugins-config-volume\") pod \"8e74da58-337e-400c-bee0-f7b5bc597904\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") "
Aug 04 09:50:59.401861 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.401717    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shmlh\" (UniqueName: \"kubernetes.io/projected/8e74da58-337e-400c-bee0-f7b5bc597904-kube-api-access-shmlh\") pod \"8e74da58-337e-400c-bee0-f7b5bc597904\" (UID: \"8e74da58-337e-400c-bee0-f7b5bc597904\") "
Aug 04 09:50:59.402019 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.401995    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e74da58-337e-400c-bee0-f7b5bc597904-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e74da58-337e-400c-bee0-f7b5bc597904" (UID: "8e74da58-337e-400c-bee0-f7b5bc597904"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:50:59.404114 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.404088    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e74da58-337e-400c-bee0-f7b5bc597904-kube-api-access-shmlh" (OuterVolumeSpecName: "kube-api-access-shmlh") pod "8e74da58-337e-400c-bee0-f7b5bc597904" (UID: "8e74da58-337e-400c-bee0-f7b5bc597904"). InnerVolumeSpecName "kube-api-access-shmlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:50:59.502868 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.502801    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e74da58-337e-400c-bee0-f7b5bc597904-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:50:59.502868 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.502825    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shmlh\" (UniqueName: \"kubernetes.io/projected/8e74da58-337e-400c-bee0-f7b5bc597904-kube-api-access-shmlh\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:50:59.983313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.983277    2558 generic.go:356] "Generic (PLEG): container finished" podID="8e74da58-337e-400c-bee0-f7b5bc597904" containerID="c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34" exitCode=0
Aug 04 09:50:59.983313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.983316    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"
Aug 04 09:50:59.983313 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.983321    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2" event={"ID":"8e74da58-337e-400c-bee0-f7b5bc597904","Type":"ContainerDied","Data":"c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34"}
Aug 04 09:50:59.983632 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.983375    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2" event={"ID":"8e74da58-337e-400c-bee0-f7b5bc597904","Type":"ContainerDied","Data":"80657b488b5e19eaf014d07f238867a69ba192bf6492a8ad38b85024a5f7ca65"}
Aug 04 09:50:59.983632 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.983405    2558 scope.go:117] "RemoveContainer" containerID="c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34"
Aug 04 09:50:59.992138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.992120    2558 scope.go:117] "RemoveContainer" containerID="c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34"
Aug 04 09:50:59.992371 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:50:59.992347    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34\": container with ID starting with c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34 not found: ID does not exist" containerID="c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34"
Aug 04 09:50:59.992427 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:50:59.992371    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34"} err="failed to get container status \"c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34\": rpc error: code = NotFound desc = could not find container \"c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34\": container with ID starting with c696cb8294e4c7696aab6b79d909c81680027e16117e0a46d337d08d3b743f34 not found: ID does not exist"
Aug 04 09:51:00.008770 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:00.008727    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"]
Aug 04 09:51:00.013825 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:00.013804    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-406b0d-66f7845596-nzlh2"]
Aug 04 09:51:01.151890 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:01.151852    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:51:01.296011 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:01.295974    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e74da58-337e-400c-bee0-f7b5bc597904" path="/var/lib/kubelet/pods/8e74da58-337e-400c-bee0-f7b5bc597904/volumes"
Aug 04 09:51:06.211592 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:06.211560    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"]
Aug 04 09:51:06.212040 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:06.211792    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy" containerID="cri-o://6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58" gracePeriod=30
Aug 04 09:51:11.469852 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.469828    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:51:11.499626 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499547    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452d27b3-a539-4efc-94f8-6df6a6c1651c-istiod-ca-cert\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499626 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499589    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-credential-socket\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499879 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499635    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-certs\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499879 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499668    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-envoy\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499879 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499707    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-data\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499879 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499753    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqqr2\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-kube-api-access-hqqr2\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499879 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499774    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-token\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499879 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499826    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-socket\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.499879 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499863    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-podinfo\") pod \"452d27b3-a539-4efc-94f8-6df6a6c1651c\" (UID: \"452d27b3-a539-4efc-94f8-6df6a6c1651c\") "
Aug 04 09:51:11.500424 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.499967    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452d27b3-a539-4efc-94f8-6df6a6c1651c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:51:11.500424 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500034    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:51:11.500424 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500239    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-data" (OuterVolumeSpecName: "istio-data") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:51:11.500424 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500419    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:51:11.500624 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500498    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:51:11.500860 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500766    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.500860 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500799    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-data\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.500860 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500812    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-workload-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.500860 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500825    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452d27b3-a539-4efc-94f8-6df6a6c1651c-istiod-ca-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.500860 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.500839    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-credential-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.503215 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.503005    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 09:51:11.503215 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.503180    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-token" (OuterVolumeSpecName: "istio-token") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:51:11.503215 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.503203    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-kube-api-access-hqqr2" (OuterVolumeSpecName: "kube-api-access-hqqr2") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "kube-api-access-hqqr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:51:11.503611 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.503589    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "452d27b3-a539-4efc-94f8-6df6a6c1651c" (UID: "452d27b3-a539-4efc-94f8-6df6a6c1651c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:51:11.601692 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.601666    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqqr2\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-kube-api-access-hqqr2\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.601692 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.601690    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-token\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.601692 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.601701    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-podinfo\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:11.601929 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:11.601710    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452d27b3-a539-4efc-94f8-6df6a6c1651c-istio-envoy\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:51:12.046122 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.046085    2558 generic.go:356] "Generic (PLEG): container finished" podID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerID="6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58" exitCode=0
Aug 04 09:51:12.046294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.046149    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"
Aug 04 09:51:12.046294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.046167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" event={"ID":"452d27b3-a539-4efc-94f8-6df6a6c1651c","Type":"ContainerDied","Data":"6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58"}
Aug 04 09:51:12.046294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.046260    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6" event={"ID":"452d27b3-a539-4efc-94f8-6df6a6c1651c","Type":"ContainerDied","Data":"02265eb61b56225397d278c2f5b0cb9014ac73bb800ff882deec767b99e44110"}
Aug 04 09:51:12.046294 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.046289    2558 scope.go:117] "RemoveContainer" containerID="6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58"
Aug 04 09:51:12.056401 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.056379    2558 scope.go:117] "RemoveContainer" containerID="d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73"
Aug 04 09:51:12.064406 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.064388    2558 scope.go:117] "RemoveContainer" containerID="6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58"
Aug 04 09:51:12.064685 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:51:12.064666    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58\": container with ID starting with 6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58 not found: ID does not exist" containerID="6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58"
Aug 04 09:51:12.064769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.064693    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58"} err="failed to get container status \"6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58\": rpc error: code = NotFound desc = could not find container \"6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58\": container with ID starting with 6c2c6079aad7e2caa2d861aaa06e1197e2defa27885a1eb791e0ccbed128cc58 not found: ID does not exist"
Aug 04 09:51:12.064769 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.064710    2558 scope.go:117] "RemoveContainer" containerID="d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73"
Aug 04 09:51:12.065018 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:51:12.065000    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73\": container with ID starting with d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73 not found: ID does not exist" containerID="d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73"
Aug 04 09:51:12.065067 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.065023    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73"} err="failed to get container status \"d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73\": rpc error: code = NotFound desc = could not find container \"d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73\": container with ID starting with d473bdcd21e6170efc3a3d397ada0c088b4120cb69e7609f71e450a24a403b73 not found: ID does not exist"
Aug 04 09:51:12.073772 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.073721    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"]
Aug 04 09:51:12.077327 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.077306    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-406b0d-openshift-default-7647747d7b-qf4g6"]
Aug 04 09:51:12.212551 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212519    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"]
Aug 04 09:51:12.212888 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212875    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da15f22f-88d2-4ec2-8cfd-5aae29273021" containerName="payload-pre-processing"
Aug 04 09:51:12.212937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212890    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="da15f22f-88d2-4ec2-8cfd-5aae29273021" containerName="payload-pre-processing"
Aug 04 09:51:12.212937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212908    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e74da58-337e-400c-bee0-f7b5bc597904" containerName="payload-processing"
Aug 04 09:51:12.212937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212914    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e74da58-337e-400c-bee0-f7b5bc597904" containerName="payload-processing"
Aug 04 09:51:12.212937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212920    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy"
Aug 04 09:51:12.212937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212926    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy"
Aug 04 09:51:12.212937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212933    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy"
Aug 04 09:51:12.212937 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212938    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy"
Aug 04 09:51:12.213152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212950    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e311bc79-87cc-444d-816c-5391109cbf23" containerName="revoke-keys"
Aug 04 09:51:12.213152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.212956    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e311bc79-87cc-444d-816c-5391109cbf23" containerName="revoke-keys"
Aug 04 09:51:12.213152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.213005    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy"
Aug 04 09:51:12.213152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.213013    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e74da58-337e-400c-bee0-f7b5bc597904" containerName="payload-processing"
Aug 04 09:51:12.213152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.213019    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e311bc79-87cc-444d-816c-5391109cbf23" containerName="revoke-keys"
Aug 04 09:51:12.213152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.213025    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" containerName="istio-proxy"
Aug 04 09:51:12.213152 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.213032    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="da15f22f-88d2-4ec2-8cfd-5aae29273021" containerName="payload-pre-processing"
Aug 04 09:51:12.217205 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.217188    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.220320 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.220300    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d27a51\""
Aug 04 09:51:12.220430 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.220342    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d27a51-dockercfg-d9zh8\""
Aug 04 09:51:12.232952 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.232927    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"]
Aug 04 09:51:12.307770 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.307664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eed7bba-2349-4e6d-9e91-639b785398f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.307770 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.307745    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtpvt\" (UniqueName: \"kubernetes.io/projected/8eed7bba-2349-4e6d-9e91-639b785398f7-kube-api-access-xtpvt\") pod \"payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.408865 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.408830    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eed7bba-2349-4e6d-9e91-639b785398f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.409046 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.408895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtpvt\" (UniqueName: \"kubernetes.io/projected/8eed7bba-2349-4e6d-9e91-639b785398f7-kube-api-access-xtpvt\") pod \"payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.409501 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.409483    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eed7bba-2349-4e6d-9e91-639b785398f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.417976 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.417955    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtpvt\" (UniqueName: \"kubernetes.io/projected/8eed7bba-2349-4e6d-9e91-639b785398f7-kube-api-access-xtpvt\") pod \"payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.527207 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.527181    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:12.665104 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:12.665077    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"]
Aug 04 09:51:12.666756 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:51:12.666715    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eed7bba_2349_4e6d_9e91_639b785398f7.slice/crio-120b216041aa536369f622b0926b09b5a730b75d673065750097ad2268c9dd3c WatchSource:0}: Error finding container 120b216041aa536369f622b0926b09b5a730b75d673065750097ad2268c9dd3c: Status 404 returned error can't find the container with id 120b216041aa536369f622b0926b09b5a730b75d673065750097ad2268c9dd3c
Aug 04 09:51:13.052578 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:13.052470    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5" event={"ID":"8eed7bba-2349-4e6d-9e91-639b785398f7","Type":"ContainerStarted","Data":"4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022"}
Aug 04 09:51:13.052578 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:13.052543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5" event={"ID":"8eed7bba-2349-4e6d-9e91-639b785398f7","Type":"ContainerStarted","Data":"120b216041aa536369f622b0926b09b5a730b75d673065750097ad2268c9dd3c"}
Aug 04 09:51:13.052578 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:13.052578    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:13.071139 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:13.071071    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5" podStartSLOduration=1.071052754 podStartE2EDuration="1.071052754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:51:13.070709426 +0000 UTC m=+2604.328666483" watchObservedRunningTime="2026-08-04 09:51:13.071052754 +0000 UTC m=+2604.329009811"
Aug 04 09:51:13.296844 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:13.296809    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="452d27b3-a539-4efc-94f8-6df6a6c1651c" path="/var/lib/kubelet/pods/452d27b3-a539-4efc-94f8-6df6a6c1651c/volumes"
Aug 04 09:51:13.844835 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:13.844801    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:51:24.062142 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:24.062109    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:51:36.120275 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.120231    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"]
Aug 04 09:51:36.124180 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.124157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.126587 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.126563    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0018e3-openshift-default-dockercfg-j8d2m\""
Aug 04 09:51:36.139044 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.139017    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"]
Aug 04 09:51:36.215646 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.215618    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-socket\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.215923 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.215652    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-certs\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.215923 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.215679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-envoy\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.215923 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.215809    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98d762b7-5b0d-4c94-b297-6432a6586548-istiod-ca-cert\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.215923 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.215869    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5t7s\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-kube-api-access-z5t7s\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.215923 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.215904    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-istio-token\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.216118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.215977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-data\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.216118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.216015    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-credential-socket\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.216118 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.216034    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98d762b7-5b0d-4c94-b297-6432a6586548-istio-podinfo\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317203 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-data\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317203 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317209    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-credential-socket\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98d762b7-5b0d-4c94-b297-6432a6586548-istio-podinfo\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-socket\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-certs\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317369    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-envoy\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317428 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98d762b7-5b0d-4c94-b297-6432a6586548-istiod-ca-cert\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317664 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5t7s\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-kube-api-access-z5t7s\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317664 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317507    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-istio-token\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317664 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317646    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-socket\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317862 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317684    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-data\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317862 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-certs\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.317997 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.317975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-credential-socket\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.318281 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.318261    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98d762b7-5b0d-4c94-b297-6432a6586548-istiod-ca-cert\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.319803 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.319778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-envoy\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.319997 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.319976    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98d762b7-5b0d-4c94-b297-6432a6586548-istio-podinfo\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.326167 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.326140    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-istio-token\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.326251 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.326230    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5t7s\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-kube-api-access-z5t7s\") pod \"e2e-api-b-0018e3-openshift-default-784957d5c6-sk899\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") " pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.435617 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.435547    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:36.568578 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.568547    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"]
Aug 04 09:51:36.571647 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:51:36.571607    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98d762b7_5b0d_4c94_b297_6432a6586548.slice/crio-473deed5344c00a948bb5781a5ca0574ae9dbb2981d86cb080678e3ede081910 WatchSource:0}: Error finding container 473deed5344c00a948bb5781a5ca0574ae9dbb2981d86cb080678e3ede081910: Status 404 returned error can't find the container with id 473deed5344c00a948bb5781a5ca0574ae9dbb2981d86cb080678e3ede081910
Aug 04 09:51:36.573800 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.573768    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:51:36.573887 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.573833    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:51:36.573887 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:36.573860    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:51:37.170048 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:37.170010    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" event={"ID":"98d762b7-5b0d-4c94-b297-6432a6586548","Type":"ContainerStarted","Data":"f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4"}
Aug 04 09:51:37.170424 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:37.170074    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" event={"ID":"98d762b7-5b0d-4c94-b297-6432a6586548","Type":"ContainerStarted","Data":"473deed5344c00a948bb5781a5ca0574ae9dbb2981d86cb080678e3ede081910"}
Aug 04 09:51:37.195112 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:37.195058    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podStartSLOduration=1.195042567 podStartE2EDuration="1.195042567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:51:37.192396678 +0000 UTC m=+2628.450353744" watchObservedRunningTime="2026-08-04 09:51:37.195042567 +0000 UTC m=+2628.452999621"
Aug 04 09:51:37.436416 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:37.436337    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:37.441412 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:37.441389    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:38.174874 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:38.174843    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:38.175776 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:38.175755    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:40.941416 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:40.941374    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"]
Aug 04 09:51:40.945667 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:40.945647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:40.948509 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:40.948485    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0018e3\""
Aug 04 09:51:40.948686 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:40.948519    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0018e3-dockercfg-spdhx\""
Aug 04 09:51:40.955538 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:40.955517    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"]
Aug 04 09:51:41.057291 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.057257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-kube-api-access-rqx7b\") pod \"payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") " pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:41.057474 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.057311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") " pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:41.158762 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.158701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-kube-api-access-rqx7b\") pod \"payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") " pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:41.158935 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.158774    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") " pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:41.159465 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.159444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") " pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:41.167442 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.167420    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-kube-api-access-rqx7b\") pod \"payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") " pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:41.258130 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.258057    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:41.399582 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:41.399556    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"]
Aug 04 09:51:41.401696 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:51:41.401670    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeddc44c8_7f06_42c9_95d8_44b9ff1fc1c9.slice/crio-d11ef617692b32a5d1627b5cb12010661e9f108233d324f8b3f1a23eeba86814 WatchSource:0}: Error finding container d11ef617692b32a5d1627b5cb12010661e9f108233d324f8b3f1a23eeba86814: Status 404 returned error can't find the container with id d11ef617692b32a5d1627b5cb12010661e9f108233d324f8b3f1a23eeba86814
Aug 04 09:51:42.196648 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:42.196617    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk" event={"ID":"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9","Type":"ContainerStarted","Data":"7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969"}
Aug 04 09:51:42.197136 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:42.196670    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk" event={"ID":"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9","Type":"ContainerStarted","Data":"d11ef617692b32a5d1627b5cb12010661e9f108233d324f8b3f1a23eeba86814"}
Aug 04 09:51:42.197136 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:42.196756    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:42.218700 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:42.218604    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk" podStartSLOduration=2.218589109 podStartE2EDuration="2.218589109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:51:42.216112056 +0000 UTC m=+2633.474069131" watchObservedRunningTime="2026-08-04 09:51:42.218589109 +0000 UTC m=+2633.476546163"
Aug 04 09:51:43.762349 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:43.762315    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:51:49.557939 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:49.557902    2558 scope.go:117] "RemoveContainer" containerID="50a9a65a61f03d39787d8ff4830512c5b60de76c8cc8d292bf845243754496cd"
Aug 04 09:51:53.176255 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.176228    2558 patch_prober.go:28] interesting pod/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Aug 04 09:51:53.176590 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.176283    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Aug 04 09:51:53.202910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.202885    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:51:53.265865 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.265833    2558 generic.go:356] "Generic (PLEG): container finished" podID="98d762b7-5b0d-4c94-b297-6432a6586548" containerID="f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4" exitCode=0
Aug 04 09:51:53.266008 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.265898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" event={"ID":"98d762b7-5b0d-4c94-b297-6432a6586548","Type":"ContainerDied","Data":"f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4"}
Aug 04 09:51:53.266279 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.266265    2558 scope.go:117] "RemoveContainer" containerID="f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4"
Aug 04 09:51:53.267437 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.267404    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:51:53.267529 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.267484    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:51:53.267529 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:53.267520    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:51:54.273320 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:54.273284    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" event={"ID":"98d762b7-5b0d-4c94-b297-6432a6586548","Type":"ContainerStarted","Data":"0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d"}
Aug 04 09:51:54.436219 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:54.436178    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:54.436615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:54.436582    2558 patch_prober.go:28] interesting pod/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Aug 04 09:51:54.436856 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:54.436829    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Aug 04 09:51:55.437000 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:55.436964    2558 patch_prober.go:28] interesting pod/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Aug 04 09:51:55.437759 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:55.437704    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Aug 04 09:51:56.436795 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:56.436723    2558 patch_prober.go:28] interesting pod/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Aug 04 09:51:56.437006 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:56.436820    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Aug 04 09:51:57.437209 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:57.436779    2558 patch_prober.go:28] interesting pod/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Aug 04 09:51:57.437209 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:57.436841    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Aug 04 09:51:58.437149 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:58.436677    2558 patch_prober.go:28] interesting pod/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Aug 04 09:51:58.437149 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:58.436764    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Aug 04 09:51:59.448311 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:59.448279    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:59.448761 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:59.448652    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:51:59.449537 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:51:59.449518    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:52:22.952461 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:22.952426    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"]
Aug 04 09:52:22.953105 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:22.952783    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5" podUID="8eed7bba-2349-4e6d-9e91-639b785398f7" containerName="payload-pre-processing" containerID="cri-o://4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022" gracePeriod=30
Aug 04 09:52:23.219526 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.219499    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:52:23.337524 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.337495    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtpvt\" (UniqueName: \"kubernetes.io/projected/8eed7bba-2349-4e6d-9e91-639b785398f7-kube-api-access-xtpvt\") pod \"8eed7bba-2349-4e6d-9e91-639b785398f7\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") "
Aug 04 09:52:23.337685 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.337562    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eed7bba-2349-4e6d-9e91-639b785398f7-plugins-config-volume\") pod \"8eed7bba-2349-4e6d-9e91-639b785398f7\" (UID: \"8eed7bba-2349-4e6d-9e91-639b785398f7\") "
Aug 04 09:52:23.337979 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.337958    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eed7bba-2349-4e6d-9e91-639b785398f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8eed7bba-2349-4e6d-9e91-639b785398f7" (UID: "8eed7bba-2349-4e6d-9e91-639b785398f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:52:23.339763 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.339708    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eed7bba-2349-4e6d-9e91-639b785398f7-kube-api-access-xtpvt" (OuterVolumeSpecName: "kube-api-access-xtpvt") pod "8eed7bba-2349-4e6d-9e91-639b785398f7" (UID: "8eed7bba-2349-4e6d-9e91-639b785398f7"). InnerVolumeSpecName "kube-api-access-xtpvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:52:23.407909 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.407881    2558 generic.go:356] "Generic (PLEG): container finished" podID="8eed7bba-2349-4e6d-9e91-639b785398f7" containerID="4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022" exitCode=0
Aug 04 09:52:23.408021 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.407926    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"
Aug 04 09:52:23.408064 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.407928    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5" event={"ID":"8eed7bba-2349-4e6d-9e91-639b785398f7","Type":"ContainerDied","Data":"4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022"}
Aug 04 09:52:23.408100 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.408064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5" event={"ID":"8eed7bba-2349-4e6d-9e91-639b785398f7","Type":"ContainerDied","Data":"120b216041aa536369f622b0926b09b5a730b75d673065750097ad2268c9dd3c"}
Aug 04 09:52:23.408134 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.408098    2558 scope.go:117] "RemoveContainer" containerID="4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022"
Aug 04 09:52:23.417288 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.417272    2558 scope.go:117] "RemoveContainer" containerID="4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022"
Aug 04 09:52:23.417550 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:52:23.417530    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022\": container with ID starting with 4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022 not found: ID does not exist" containerID="4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022"
Aug 04 09:52:23.417596 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.417560    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022"} err="failed to get container status \"4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022\": rpc error: code = NotFound desc = could not find container \"4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022\": container with ID starting with 4d8b9e5bf98112ebf5599a7604ad9210dd8139ff8b7110c3aab874a32f744022 not found: ID does not exist"
Aug 04 09:52:23.431134 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.431107    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"]
Aug 04 09:52:23.433765 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.433729    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d27a51-65ddd97496-hrjx5"]
Aug 04 09:52:23.438556 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.438538    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtpvt\" (UniqueName: \"kubernetes.io/projected/8eed7bba-2349-4e6d-9e91-639b785398f7-kube-api-access-xtpvt\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:23.438610 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:23.438559    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eed7bba-2349-4e6d-9e91-639b785398f7-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:24.846005 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:24.845970    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:52:25.294961 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:25.294883    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eed7bba-2349-4e6d-9e91-639b785398f7" path="/var/lib/kubelet/pods/8eed7bba-2349-4e6d-9e91-639b785398f7/volumes"
Aug 04 09:52:45.773202 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:45.773155    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"]
Aug 04 09:52:45.773708 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:45.773437    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk" podUID="eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" containerName="payload-processing" containerID="cri-o://7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969" gracePeriod=30
Aug 04 09:52:46.024958 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.024892    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:52:46.131998 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.131964    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-kube-api-access-rqx7b\") pod \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") "
Aug 04 09:52:46.132160 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.132090    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-plugins-config-volume\") pod \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\" (UID: \"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9\") "
Aug 04 09:52:46.132478 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.132446    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" (UID: "eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:52:46.134085 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.134060    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-kube-api-access-rqx7b" (OuterVolumeSpecName: "kube-api-access-rqx7b") pod "eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" (UID: "eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9"). InnerVolumeSpecName "kube-api-access-rqx7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:52:46.233558 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.233530    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:46.233558 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.233555    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqx7b\" (UniqueName: \"kubernetes.io/projected/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9-kube-api-access-rqx7b\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:46.519851 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.519816    2558 generic.go:356] "Generic (PLEG): container finished" podID="eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" containerID="7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969" exitCode=0
Aug 04 09:52:46.520038 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.519871    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"
Aug 04 09:52:46.520038 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.519907    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk" event={"ID":"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9","Type":"ContainerDied","Data":"7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969"}
Aug 04 09:52:46.520038 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.519964    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk" event={"ID":"eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9","Type":"ContainerDied","Data":"d11ef617692b32a5d1627b5cb12010661e9f108233d324f8b3f1a23eeba86814"}
Aug 04 09:52:46.520038 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.519995    2558 scope.go:117] "RemoveContainer" containerID="7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969"
Aug 04 09:52:46.528891 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.528864    2558 scope.go:117] "RemoveContainer" containerID="7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969"
Aug 04 09:52:46.529159 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:52:46.529138    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969\": container with ID starting with 7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969 not found: ID does not exist" containerID="7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969"
Aug 04 09:52:46.529223 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.529170    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969"} err="failed to get container status \"7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969\": rpc error: code = NotFound desc = could not find container \"7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969\": container with ID starting with 7032c8a6c1a7ad7ec5b21491ae3379314aa6ef281c5ae2f9c7f4d3819c258969 not found: ID does not exist"
Aug 04 09:52:46.542859 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.542826    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"]
Aug 04 09:52:46.548386 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:46.548363    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0018e3-6bcdc78556-tpqhk"]
Aug 04 09:52:47.295441 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:47.295410    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" path="/var/lib/kubelet/pods/eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9/volumes"
Aug 04 09:52:47.941947 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:47.941915    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:52:52.888557 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:52.888521    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"]
Aug 04 09:52:52.889054 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:52.888726    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy" containerID="cri-o://0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d" gracePeriod=30
Aug 04 09:52:58.160877 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.160816    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:52:58.232940 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.232908    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-certs\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.232950    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98d762b7-5b0d-4c94-b297-6432a6586548-istio-podinfo\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.232991    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-socket\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233012    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-credential-socket\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233066    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-istio-token\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233138 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233125    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-envoy\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233407 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233152    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-data\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233407 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233168    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:52:58.233407 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233181    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98d762b7-5b0d-4c94-b297-6432a6586548-istiod-ca-cert\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233407 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233230    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5t7s\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-kube-api-access-z5t7s\") pod \"98d762b7-5b0d-4c94-b297-6432a6586548\" (UID: \"98d762b7-5b0d-4c94-b297-6432a6586548\") "
Aug 04 09:52:58.233407 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233317    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:52:58.233407 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233364    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:52:58.233709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233503    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-data" (OuterVolumeSpecName: "istio-data") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:52:58.233709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233560    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98d762b7-5b0d-4c94-b297-6432a6586548-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 09:52:58.233709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233636    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.233709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233651    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-workload-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.233709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233665    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-credential-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.233709 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.233680    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-data\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.235606 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.235578    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98d762b7-5b0d-4c94-b297-6432a6586548-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 09:52:58.235830 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.235809    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-istio-token" (OuterVolumeSpecName: "istio-token") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:52:58.236129 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.236104    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 09:52:58.236210 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.236160    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-kube-api-access-z5t7s" (OuterVolumeSpecName: "kube-api-access-z5t7s") pod "98d762b7-5b0d-4c94-b297-6432a6586548" (UID: "98d762b7-5b0d-4c94-b297-6432a6586548"). InnerVolumeSpecName "kube-api-access-z5t7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 09:52:58.334882 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.334846    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-istio-token\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.334882 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.334869    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98d762b7-5b0d-4c94-b297-6432a6586548-istio-envoy\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.334882 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.334878    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98d762b7-5b0d-4c94-b297-6432a6586548-istiod-ca-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.334882 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.334886    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5t7s\" (UniqueName: \"kubernetes.io/projected/98d762b7-5b0d-4c94-b297-6432a6586548-kube-api-access-z5t7s\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.335184 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.334896    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98d762b7-5b0d-4c94-b297-6432a6586548-istio-podinfo\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 09:52:58.580023 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.579983    2558 generic.go:356] "Generic (PLEG): container finished" podID="98d762b7-5b0d-4c94-b297-6432a6586548" containerID="0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d" exitCode=0
Aug 04 09:52:58.580185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.580034    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"
Aug 04 09:52:58.580185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.580060    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" event={"ID":"98d762b7-5b0d-4c94-b297-6432a6586548","Type":"ContainerDied","Data":"0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d"}
Aug 04 09:52:58.580185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.580116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899" event={"ID":"98d762b7-5b0d-4c94-b297-6432a6586548","Type":"ContainerDied","Data":"473deed5344c00a948bb5781a5ca0574ae9dbb2981d86cb080678e3ede081910"}
Aug 04 09:52:58.580185 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.580138    2558 scope.go:117] "RemoveContainer" containerID="0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d"
Aug 04 09:52:58.591759 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.591097    2558 scope.go:117] "RemoveContainer" containerID="f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4"
Aug 04 09:52:58.601107 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.601066    2558 scope.go:117] "RemoveContainer" containerID="0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d"
Aug 04 09:52:58.601419 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:52:58.601397    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d\": container with ID starting with 0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d not found: ID does not exist" containerID="0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d"
Aug 04 09:52:58.601518 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.601430    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d"} err="failed to get container status \"0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d\": rpc error: code = NotFound desc = could not find container \"0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d\": container with ID starting with 0064a3fdd366609d8610f19a3e53bf89672f43f0f0454639c53e48989e377e0d not found: ID does not exist"
Aug 04 09:52:58.601518 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.601454    2558 scope.go:117] "RemoveContainer" containerID="f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4"
Aug 04 09:52:58.601803 ip-10-0-143-106 kubenswrapper[2558]: E0804 09:52:58.601777    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4\": container with ID starting with f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4 not found: ID does not exist" containerID="f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4"
Aug 04 09:52:58.601895 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.601813    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4"} err="failed to get container status \"f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4\": rpc error: code = NotFound desc = could not find container \"f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4\": container with ID starting with f2698dd4f117cf0f0b46e68568b62a11cf59a330e7bbdb2b5b0ea47bac0aded4 not found: ID does not exist"
Aug 04 09:52:58.611727 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.611697    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"]
Aug 04 09:52:58.613254 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.613232    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0018e3-openshift-default-784957d5c6-sk899"]
Aug 04 09:52:58.837028 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.836918    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"]
Aug 04 09:52:58.837342 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837329    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy"
Aug 04 09:52:58.837389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837344    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy"
Aug 04 09:52:58.837389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837354    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy"
Aug 04 09:52:58.837389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837360    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy"
Aug 04 09:52:58.837389 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837385    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" containerName="payload-processing"
Aug 04 09:52:58.837517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837393    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" containerName="payload-processing"
Aug 04 09:52:58.837517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837400    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eed7bba-2349-4e6d-9e91-639b785398f7" containerName="payload-pre-processing"
Aug 04 09:52:58.837517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837405    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eed7bba-2349-4e6d-9e91-639b785398f7" containerName="payload-pre-processing"
Aug 04 09:52:58.837517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837465    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy"
Aug 04 09:52:58.837517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837477    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eed7bba-2349-4e6d-9e91-639b785398f7" containerName="payload-pre-processing"
Aug 04 09:52:58.837517 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.837483    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eddc44c8-7f06-42c9-95d8-44b9ff1fc1c9" containerName="payload-processing"
Aug 04 09:52:58.841956 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.841936    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:58.845498 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.845476    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-497679\""
Aug 04 09:52:58.845637 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.845500    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-497679-dockercfg-wkf5h\""
Aug 04 09:52:58.852066 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.852041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"]
Aug 04 09:52:58.939975 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.939937    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7qgg\" (UniqueName: \"kubernetes.io/projected/a78d9c91-20d3-4b2b-b9de-1e522923266f-kube-api-access-c7qgg\") pod \"payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:58.940255 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:58.939992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a78d9c91-20d3-4b2b-b9de-1e522923266f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:59.040692 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.040654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7qgg\" (UniqueName: \"kubernetes.io/projected/a78d9c91-20d3-4b2b-b9de-1e522923266f-kube-api-access-c7qgg\") pod \"payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:59.040692 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.040694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a78d9c91-20d3-4b2b-b9de-1e522923266f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:59.041616 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.041584    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a78d9c91-20d3-4b2b-b9de-1e522923266f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:59.050764 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.050694    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7qgg\" (UniqueName: \"kubernetes.io/projected/a78d9c91-20d3-4b2b-b9de-1e522923266f-kube-api-access-c7qgg\") pod \"payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:59.153893 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.153816    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:59.293908 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:52:59.293879    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda78d9c91_20d3_4b2b_b9de_1e522923266f.slice/crio-ae0d82827c5ebe47d4f1abfce2496dcb4f8356724bbfe6c24bef8c6716681133 WatchSource:0}: Error finding container ae0d82827c5ebe47d4f1abfce2496dcb4f8356724bbfe6c24bef8c6716681133: Status 404 returned error can't find the container with id ae0d82827c5ebe47d4f1abfce2496dcb4f8356724bbfe6c24bef8c6716681133
Aug 04 09:52:59.297713 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.297690    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" path="/var/lib/kubelet/pods/98d762b7-5b0d-4c94-b297-6432a6586548/volumes"
Aug 04 09:52:59.298005 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.297991    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"]
Aug 04 09:52:59.587416 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.587385    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf" event={"ID":"a78d9c91-20d3-4b2b-b9de-1e522923266f","Type":"ContainerStarted","Data":"a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2"}
Aug 04 09:52:59.587615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.587445    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:52:59.587615 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.587475    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf" event={"ID":"a78d9c91-20d3-4b2b-b9de-1e522923266f","Type":"ContainerStarted","Data":"ae0d82827c5ebe47d4f1abfce2496dcb4f8356724bbfe6c24bef8c6716681133"}
Aug 04 09:52:59.606076 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:52:59.606028    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf" podStartSLOduration=1.606013259 podStartE2EDuration="1.606013259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:52:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:52:59.604101961 +0000 UTC m=+2710.862059223" watchObservedRunningTime="2026-08-04 09:52:59.606013259 +0000 UTC m=+2710.863970312"
Aug 04 09:53:00.654267 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:00.654231    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:53:10.595620 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:10.595589    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 09:53:22.641477 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.641446    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"]
Aug 04 09:53:22.641972 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.641894    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="98d762b7-5b0d-4c94-b297-6432a6586548" containerName="istio-proxy"
Aug 04 09:53:22.645179 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.645142    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.649890 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.649870    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b0bc81-openshift-default-dockercfg-hd9z9\""
Aug 04 09:53:22.668943 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.668899    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"]
Aug 04 09:53:22.743668 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.743630    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-certs\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.743985 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.743939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-envoy\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.744112 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.744000    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-istio-token\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.744112 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.744039    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e55b90-84d2-418e-8718-817731535693-istio-podinfo\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.744112 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.744074    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8wxp\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-kube-api-access-m8wxp\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.744112 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.744102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-data\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.744333 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.744129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-socket\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.744333 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.744162    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-credential-socket\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.744333 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.744181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e55b90-84d2-418e-8718-817731535693-istiod-ca-cert\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845121 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845083    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e55b90-84d2-418e-8718-817731535693-istio-podinfo\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845121 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8wxp\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-kube-api-access-m8wxp\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845151    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-data\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-socket\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845201    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-credential-socket\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845218    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e55b90-84d2-418e-8718-817731535693-istiod-ca-cert\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845304    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-certs\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-envoy\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845384 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-istio-token\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845788 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845665    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-socket\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845788 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845704    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-credential-socket\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845914 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845799    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-data\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.845914 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.845836    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-certs\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.846141 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.846119    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e55b90-84d2-418e-8718-817731535693-istiod-ca-cert\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.847851 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.847816    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e55b90-84d2-418e-8718-817731535693-istio-podinfo\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.847971 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.847867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-envoy\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.855228 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.855199    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8wxp\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-kube-api-access-m8wxp\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.855386 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.855351    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-istio-token\") pod \"e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") " pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:22.956658 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:22.956573    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:23.104781 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.104755    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"]
Aug 04 09:53:23.107274 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:53:23.107247    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e55b90_84d2_418e_8718_817731535693.slice/crio-31b849063cd826fff2c5c04abd8e6465ce2becbae9579aa833dc1aa185335a76 WatchSource:0}: Error finding container 31b849063cd826fff2c5c04abd8e6465ce2becbae9579aa833dc1aa185335a76: Status 404 returned error can't find the container with id 31b849063cd826fff2c5c04abd8e6465ce2becbae9579aa833dc1aa185335a76
Aug 04 09:53:23.109396 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.109364    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:53:23.109492 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.109442    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:53:23.109555 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.109487    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:53:23.708093 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.708059    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" event={"ID":"d0e55b90-84d2-418e-8718-817731535693","Type":"ContainerStarted","Data":"efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5"}
Aug 04 09:53:23.708466 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.708114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" event={"ID":"d0e55b90-84d2-418e-8718-817731535693","Type":"ContainerStarted","Data":"31b849063cd826fff2c5c04abd8e6465ce2becbae9579aa833dc1aa185335a76"}
Aug 04 09:53:23.738985 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.738931    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podStartSLOduration=1.738915593 podStartE2EDuration="1.738915593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:53:23.73636981 +0000 UTC m=+2734.994326865" watchObservedRunningTime="2026-08-04 09:53:23.738915593 +0000 UTC m=+2734.996872648"
Aug 04 09:53:23.957382 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:23.957353    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:24.962680 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:24.962651    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:25.718176 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:25.718146    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:25.719134 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:25.719117    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:27.915910 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:27.915867    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"]
Aug 04 09:53:27.920315 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:27.920291    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:27.927051 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:27.927030    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-b0bc81\""
Aug 04 09:53:27.929029 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:27.928815    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-b0bc81-dockercfg-gjfhs\""
Aug 04 09:53:27.947061 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:27.947022    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"]
Aug 04 09:53:28.091029 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.090978    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.091029 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.091033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vbhl\" (UniqueName: \"kubernetes.io/projected/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-kube-api-access-5vbhl\") pod \"payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.191941 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.191855    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.191941 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.191893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vbhl\" (UniqueName: \"kubernetes.io/projected/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-kube-api-access-5vbhl\") pod \"payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.192572 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.192554    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.202513 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.202489    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vbhl\" (UniqueName: \"kubernetes.io/projected/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-kube-api-access-5vbhl\") pod \"payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.232548 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.232499    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.602130 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.602109    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"]
Aug 04 09:53:28.605065 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:53:28.605033    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f53787b_8a2f_4c4f_9317_d709c7efbc5f.slice/crio-02bc25b09d3085426edeb1f11237652d897a4243857c4da63dc381dfbbbbb202 WatchSource:0}: Error finding container 02bc25b09d3085426edeb1f11237652d897a4243857c4da63dc381dfbbbbb202: Status 404 returned error can't find the container with id 02bc25b09d3085426edeb1f11237652d897a4243857c4da63dc381dfbbbbb202
Aug 04 09:53:28.733787 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.733744    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp" event={"ID":"2f53787b-8a2f-4c4f-9317-d709c7efbc5f","Type":"ContainerStarted","Data":"0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7"}
Aug 04 09:53:28.733960 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.733808    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp" event={"ID":"2f53787b-8a2f-4c4f-9317-d709c7efbc5f","Type":"ContainerStarted","Data":"02bc25b09d3085426edeb1f11237652d897a4243857c4da63dc381dfbbbbb202"}
Aug 04 09:53:28.733960 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.733847    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:28.763811 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:28.763757    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp" podStartSLOduration=1.763730349 podStartE2EDuration="1.763730349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:53:28.761140452 +0000 UTC m=+2740.019097521" watchObservedRunningTime="2026-08-04 09:53:28.763730349 +0000 UTC m=+2740.021687404"
Aug 04 09:53:30.488004 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:30.487970    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:53:39.741204 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:39.741174    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 09:53:40.719144 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.719110    2558 patch_prober.go:28] interesting pod/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Aug 04 09:53:40.719347 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.719177    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Aug 04 09:53:40.796848 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.796816    2558 generic.go:356] "Generic (PLEG): container finished" podID="d0e55b90-84d2-418e-8718-817731535693" containerID="efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5" exitCode=0
Aug 04 09:53:40.797224 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.796887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" event={"ID":"d0e55b90-84d2-418e-8718-817731535693","Type":"ContainerDied","Data":"efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5"}
Aug 04 09:53:40.797360 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.797344    2558 scope.go:117] "RemoveContainer" containerID="efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5"
Aug 04 09:53:40.798411 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.798384    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:53:40.798496 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.798450    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:53:40.798496 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:40.798483    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 09:53:41.803654 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:41.803619    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" event={"ID":"d0e55b90-84d2-418e-8718-817731535693","Type":"ContainerStarted","Data":"a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded"}
Aug 04 09:53:41.956902 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:41.956871    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:41.957116 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:41.957088    2558 patch_prober.go:28] interesting pod/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Aug 04 09:53:41.957165 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:41.957145    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Aug 04 09:53:42.957706 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:42.957661    2558 patch_prober.go:28] interesting pod/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Aug 04 09:53:42.958210 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:42.957725    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Aug 04 09:53:43.957984 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:43.957942    2558 patch_prober.go:28] interesting pod/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Aug 04 09:53:43.958461 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:43.958001    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Aug 04 09:53:44.959716 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:44.959561    2558 patch_prober.go:28] interesting pod/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Aug 04 09:53:44.960189 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:44.959761    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Aug 04 09:53:45.957264 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:45.957230    2558 patch_prober.go:28] interesting pod/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Aug 04 09:53:45.957480 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:45.957308    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Aug 04 09:53:46.961271 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:46.961239    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:46.961751 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:46.961668    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:53:46.962369 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:53:46.962350    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 09:54:05.448032 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:54:05.447997    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:54:40.271716 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:54:40.271678    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:55:06.657675 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:55:06.657645    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:55:20.254257 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:55:20.254218    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:55:31.551011 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:55:31.550972    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:55:46.853199 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:55:46.853157    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:55:58.877130 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:55:58.877095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:56:13.371327 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:56:13.371289    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:56:25.346679 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:56:25.346649    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:56:40.035050 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:56:40.035016    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:56:49.620301 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:56:49.620271    2558 scope.go:117] "RemoveContainer" containerID="e309ec8720998aace51fd78ec630cb020aa49257969313dbf17adba2b75d0d90"
Aug 04 09:56:51.688053 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:56:51.688023    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:57:06.754177 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:57:06.754142    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:57:18.443790 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:57:18.443684    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:57:33.449286 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:57:33.449252    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:57:42.142577 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:57:42.142541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:58:07.464816 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:07.464778    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:58:40.763718 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.763683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"]
Aug 04 09:58:40.767146 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.767111    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.769967 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.769946    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"kube-root-ca.crt\""
Aug 04 09:58:40.770340 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.770321    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"openshift-service-ca.crt\""
Aug 04 09:58:40.771217 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.771198    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"sub-test-model-b0bc81-kserve-self-signed-certs\""
Aug 04 09:58:40.771302 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.771199    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"default-dockercfg-jb7w9\""
Aug 04 09:58:40.779920 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.779900    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"]
Aug 04 09:58:40.895487 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.895443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-dshm\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.895823 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.895803    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-tmp-dir\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.895967 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.895945    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a989d00-ae84-4c60-a3da-290456563d77-tls-certs\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.896099 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.895996    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqltc\" (UniqueName: \"kubernetes.io/projected/7a989d00-ae84-4c60-a3da-290456563d77-kube-api-access-cqltc\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.896099 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.896027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-home\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.896099 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.896087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-model-cache\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.996566 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.996530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-tmp-dir\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.996566 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.996564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a989d00-ae84-4c60-a3da-290456563d77-tls-certs\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.996855 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.996599    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqltc\" (UniqueName: \"kubernetes.io/projected/7a989d00-ae84-4c60-a3da-290456563d77-kube-api-access-cqltc\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.996855 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.996722    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-home\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.996855 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.996826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-model-cache\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.997005 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.996910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-dshm\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.997042 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.997009    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-tmp-dir\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.997079 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.997060    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-home\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.997200 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.997176    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-model-cache\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.999141 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.999114    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-dshm\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:40.999436 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:40.999413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a989d00-ae84-4c60-a3da-290456563d77-tls-certs\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:41.008712 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:41.008689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqltc\" (UniqueName: \"kubernetes.io/projected/7a989d00-ae84-4c60-a3da-290456563d77-kube-api-access-cqltc\") pod \"sub-test-model-b0bc81-kserve-5b446f5979-s2l6d\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") " pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:41.077648 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:41.077578    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:41.230508 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:41.230439    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"]
Aug 04 09:58:41.232717 ip-10-0-143-106 kubenswrapper[2558]: W0804 09:58:41.232690    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a989d00_ae84_4c60_a3da_290456563d77.slice/crio-6f424a21d569604f7469b4958216e167867f59d6c738ff325eed9a84d7d43250 WatchSource:0}: Error finding container 6f424a21d569604f7469b4958216e167867f59d6c738ff325eed9a84d7d43250: Status 404 returned error can't find the container with id 6f424a21d569604f7469b4958216e167867f59d6c738ff325eed9a84d7d43250
Aug 04 09:58:42.222946 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:42.222901    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d" event={"ID":"7a989d00-ae84-4c60-a3da-290456563d77","Type":"ContainerStarted","Data":"a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a"}
Aug 04 09:58:42.222946 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:42.222942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d" event={"ID":"7a989d00-ae84-4c60-a3da-290456563d77","Type":"ContainerStarted","Data":"6f424a21d569604f7469b4958216e167867f59d6c738ff325eed9a84d7d43250"}
Aug 04 09:58:42.247996 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:42.247949    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d" podStartSLOduration=2.247932258 podStartE2EDuration="2.247932258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:58:42.245160076 +0000 UTC m=+3053.503117131" watchObservedRunningTime="2026-08-04 09:58:42.247932258 +0000 UTC m=+3053.505889342"
Aug 04 09:58:43.345532 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:43.345503    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:58:51.078583 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:51.078543    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:51.079062 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:51.078606    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:51.090974 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:51.090956    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:58:51.279549 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:58:51.279518    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 09:59:17.153907 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:59:17.153870    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:59:27.862359 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:59:27.862324    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 09:59:56.238170 ip-10-0-143-106 kubenswrapper[2558]: I0804 09:59:56.238128    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:00:00.157394 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.157359    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q"]
Aug 04 10:00:00.160800 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.160781    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q"
Aug 04 10:00:00.165526 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.165504    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\""
Aug 04 10:00:00.173014 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.172986    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q"]
Aug 04 10:00:00.173127 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.172996    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q"
Aug 04 10:00:00.317048 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.317023    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q"]
Aug 04 10:00:00.319675 ip-10-0-143-106 kubenswrapper[2558]: W0804 10:00:00.319651    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e613e08_5e9d_4bbe_8679_0a81ece1e3f1.slice/crio-b93b7e050bac3c9a941c3d76c1f0fb1837b537cd734f25e79e8f6cb07e41fe57 WatchSource:0}: Error finding container b93b7e050bac3c9a941c3d76c1f0fb1837b537cd734f25e79e8f6cb07e41fe57: Status 404 returned error can't find the container with id b93b7e050bac3c9a941c3d76c1f0fb1837b537cd734f25e79e8f6cb07e41fe57
Aug 04 10:00:00.590362 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.590335    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q" event={"ID":"9e613e08-5e9d-4bbe-8679-0a81ece1e3f1","Type":"ContainerStarted","Data":"82db53450dc4e0db839de630fc81f1ca65645877e52c0155abe0dbf33d1b9227"}
Aug 04 10:00:00.590492 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.590383    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q" event={"ID":"9e613e08-5e9d-4bbe-8679-0a81ece1e3f1","Type":"ContainerStarted","Data":"b93b7e050bac3c9a941c3d76c1f0fb1837b537cd734f25e79e8f6cb07e41fe57"}
Aug 04 10:00:00.607466 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:00.607418    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q" podStartSLOduration=0.607401471 podStartE2EDuration="607.401471ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:00:00.606324893 +0000 UTC m=+3131.864281945" watchObservedRunningTime="2026-08-04 10:00:00.607401471 +0000 UTC m=+3131.865358525"
Aug 04 10:00:01.596280 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:01.596244    2558 generic.go:356] "Generic (PLEG): container finished" podID="9e613e08-5e9d-4bbe-8679-0a81ece1e3f1" containerID="82db53450dc4e0db839de630fc81f1ca65645877e52c0155abe0dbf33d1b9227" exitCode=0
Aug 04 10:00:01.596669 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:01.596328    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q" event={"ID":"9e613e08-5e9d-4bbe-8679-0a81ece1e3f1","Type":"ContainerDied","Data":"82db53450dc4e0db839de630fc81f1ca65645877e52c0155abe0dbf33d1b9227"}
Aug 04 10:00:02.245943 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:02.245908    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"]
Aug 04 10:00:02.745980 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:02.745955    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q"
Aug 04 10:00:03.607583 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:03.607549    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q" event={"ID":"9e613e08-5e9d-4bbe-8679-0a81ece1e3f1","Type":"ContainerDied","Data":"b93b7e050bac3c9a941c3d76c1f0fb1837b537cd734f25e79e8f6cb07e41fe57"}
Aug 04 10:00:03.607583 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:03.607575    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763960-xsp8q"
Aug 04 10:00:03.607810 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:03.607622    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b93b7e050bac3c9a941c3d76c1f0fb1837b537cd734f25e79e8f6cb07e41fe57"
Aug 04 10:00:07.162994 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:07.162957    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:00:17.265151 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.265063    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d" podUID="7a989d00-ae84-4c60-a3da-290456563d77" containerName="main" containerID="cri-o://a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a" gracePeriod=45
Aug 04 10:00:17.526500 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.526439    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 10:00:17.630696 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.630663    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a989d00-ae84-4c60-a3da-290456563d77-tls-certs\") pod \"7a989d00-ae84-4c60-a3da-290456563d77\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") "
Aug 04 10:00:17.630940 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.630703    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqltc\" (UniqueName: \"kubernetes.io/projected/7a989d00-ae84-4c60-a3da-290456563d77-kube-api-access-cqltc\") pod \"7a989d00-ae84-4c60-a3da-290456563d77\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") "
Aug 04 10:00:17.630940 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.630725    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-model-cache\") pod \"7a989d00-ae84-4c60-a3da-290456563d77\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") "
Aug 04 10:00:17.630940 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.630761    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-dshm\") pod \"7a989d00-ae84-4c60-a3da-290456563d77\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") "
Aug 04 10:00:17.630940 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.630814    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-home\") pod \"7a989d00-ae84-4c60-a3da-290456563d77\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") "
Aug 04 10:00:17.630940 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.630907    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-tmp-dir\") pod \"7a989d00-ae84-4c60-a3da-290456563d77\" (UID: \"7a989d00-ae84-4c60-a3da-290456563d77\") "
Aug 04 10:00:17.631227 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.630996    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a989d00-ae84-4c60-a3da-290456563d77" (UID: "7a989d00-ae84-4c60-a3da-290456563d77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:00:17.631227 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.631183    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-home" (OuterVolumeSpecName: "home") pod "7a989d00-ae84-4c60-a3da-290456563d77" (UID: "7a989d00-ae84-4c60-a3da-290456563d77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:00:17.631227 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.631207    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a989d00-ae84-4c60-a3da-290456563d77" (UID: "7a989d00-ae84-4c60-a3da-290456563d77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:00:17.631392 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.631304    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-tmp-dir\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:00:17.631392 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.631323    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-model-cache\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:00:17.631392 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.631336    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-home\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:00:17.633121 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.633090    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a989d00-ae84-4c60-a3da-290456563d77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a989d00-ae84-4c60-a3da-290456563d77" (UID: "7a989d00-ae84-4c60-a3da-290456563d77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 10:00:17.633274 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.633253    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-dshm" (OuterVolumeSpecName: "dshm") pod "7a989d00-ae84-4c60-a3da-290456563d77" (UID: "7a989d00-ae84-4c60-a3da-290456563d77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:00:17.633345 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.633286    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a989d00-ae84-4c60-a3da-290456563d77-kube-api-access-cqltc" (OuterVolumeSpecName: "kube-api-access-cqltc") pod "7a989d00-ae84-4c60-a3da-290456563d77" (UID: "7a989d00-ae84-4c60-a3da-290456563d77"). InnerVolumeSpecName "kube-api-access-cqltc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:00:17.677800 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.677759    2558 generic.go:356] "Generic (PLEG): container finished" podID="7a989d00-ae84-4c60-a3da-290456563d77" containerID="a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a" exitCode=0
Aug 04 10:00:17.677800 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.677799    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"
Aug 04 10:00:17.677998 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.677824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d" event={"ID":"7a989d00-ae84-4c60-a3da-290456563d77","Type":"ContainerDied","Data":"a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a"}
Aug 04 10:00:17.677998 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.677864    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d" event={"ID":"7a989d00-ae84-4c60-a3da-290456563d77","Type":"ContainerDied","Data":"6f424a21d569604f7469b4958216e167867f59d6c738ff325eed9a84d7d43250"}
Aug 04 10:00:17.677998 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.677882    2558 scope.go:117] "RemoveContainer" containerID="a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a"
Aug 04 10:00:17.687490 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.687474    2558 scope.go:117] "RemoveContainer" containerID="a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a"
Aug 04 10:00:17.687796 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:00:17.687772    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a\": container with ID starting with a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a not found: ID does not exist" containerID="a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a"
Aug 04 10:00:17.687893 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.687804    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a"} err="failed to get container status \"a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a\": rpc error: code = NotFound desc = could not find container \"a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a\": container with ID starting with a84e380f2bbf8154da5d1ccc7b2d70cc672dec883f3acacac577e769ee00e90a not found: ID does not exist"
Aug 04 10:00:17.707503 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.707475    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"]
Aug 04 10:00:17.712524 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.712504    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/sub-test-model-b0bc81-kserve-5b446f5979-s2l6d"]
Aug 04 10:00:17.732024 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.731999    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a989d00-ae84-4c60-a3da-290456563d77-tls-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:00:17.732024 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.732023    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqltc\" (UniqueName: \"kubernetes.io/projected/7a989d00-ae84-4c60-a3da-290456563d77-kube-api-access-cqltc\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:00:17.732156 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:17.732034    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a989d00-ae84-4c60-a3da-290456563d77-dshm\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:00:19.295917 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:19.295884    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a989d00-ae84-4c60-a3da-290456563d77" path="/var/lib/kubelet/pods/7a989d00-ae84-4c60-a3da-290456563d77/volumes"
Aug 04 10:00:24.447343 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:24.447306    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:00:39.854915 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.854879    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"]
Aug 04 10:00:39.855407 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.855393    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a989d00-ae84-4c60-a3da-290456563d77" containerName="main"
Aug 04 10:00:39.855452 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.855410    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a989d00-ae84-4c60-a3da-290456563d77" containerName="main"
Aug 04 10:00:39.855452 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.855431    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e613e08-5e9d-4bbe-8679-0a81ece1e3f1" containerName="cleanup"
Aug 04 10:00:39.855452 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.855440    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e613e08-5e9d-4bbe-8679-0a81ece1e3f1" containerName="cleanup"
Aug 04 10:00:39.855547 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.855520    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a989d00-ae84-4c60-a3da-290456563d77" containerName="main"
Aug 04 10:00:39.855547 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.855531    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e613e08-5e9d-4bbe-8679-0a81ece1e3f1" containerName="cleanup"
Aug 04 10:00:39.858875 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.858853    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:39.861727 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.861702    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"kube-root-ca.crt\""
Aug 04 10:00:39.862526 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.862485    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"openshift-service-ca.crt\""
Aug 04 10:00:39.862662 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.862520    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"rate-test-model-b0bc81-kserve-self-signed-certs\""
Aug 04 10:00:39.862662 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.862562    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b0bc81\"/\"default-dockercfg-jb7w9\""
Aug 04 10:00:39.873575 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.873557    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"]
Aug 04 10:00:39.917239 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.917208    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tls-certs\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:39.917376 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.917256    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-model-cache\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:39.917376 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.917318    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tmp-dir\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:39.917376 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.917355    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-home\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:39.917376 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.917370    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-dshm\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:39.917517 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:39.917388    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh4lb\" (UniqueName: \"kubernetes.io/projected/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-kube-api-access-kh4lb\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.017964 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.017930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh4lb\" (UniqueName: \"kubernetes.io/projected/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-kube-api-access-kh4lb\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018158 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018028    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tls-certs\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018158 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018073    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-model-cache\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018158 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tmp-dir\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018158 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018157    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-home\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018388 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-dshm\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018580 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-model-cache\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018698 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-home\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.018952 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.018930    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tmp-dir\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.020757 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.020708    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-dshm\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.021027 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.020995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tls-certs\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.026611 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.026591    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh4lb\" (UniqueName: \"kubernetes.io/projected/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-kube-api-access-kh4lb\") pod \"rate-test-model-b0bc81-kserve-7d994f77f5-gjqds\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") " pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.169283 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.169193    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:40.305578 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.305552    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"]
Aug 04 10:00:40.308356 ip-10-0-143-106 kubenswrapper[2558]: W0804 10:00:40.308327    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ac9e57f_1a60_4a28_8b76_064ae871b6c9.slice/crio-db299f7604a25e4b3c07454958121690aa37432c6976f0fa664d6887ed7a09e3 WatchSource:0}: Error finding container db299f7604a25e4b3c07454958121690aa37432c6976f0fa664d6887ed7a09e3: Status 404 returned error can't find the container with id db299f7604a25e4b3c07454958121690aa37432c6976f0fa664d6887ed7a09e3
Aug 04 10:00:40.797211 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.797165    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds" event={"ID":"2ac9e57f-1a60-4a28-8b76-064ae871b6c9","Type":"ContainerStarted","Data":"1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd"}
Aug 04 10:00:40.797381 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.797220    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds" event={"ID":"2ac9e57f-1a60-4a28-8b76-064ae871b6c9","Type":"ContainerStarted","Data":"db299f7604a25e4b3c07454958121690aa37432c6976f0fa664d6887ed7a09e3"}
Aug 04 10:00:40.820557 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:40.820507    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds" podStartSLOduration=1.820492327 podStartE2EDuration="1.820492327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:00:40.820121876 +0000 UTC m=+3172.078078932" watchObservedRunningTime="2026-08-04 10:00:40.820492327 +0000 UTC m=+3172.078449382"
Aug 04 10:00:42.441789 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:42.441749    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:00:50.169499 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:50.169466    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:50.169499 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:50.169507    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:50.181982 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:50.181958    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:00:50.854321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:00:50.854296    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:01:19.765845 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:01:19.765810    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:01:28.146424 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:01:28.146390    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:01:44.243946 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:01:44.243867    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:01:50.556313 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:01:50.556278    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"]
Aug 04 10:01:53.067789 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:01:53.067752    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:02:05.575479 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.575405    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds" podUID="2ac9e57f-1a60-4a28-8b76-064ae871b6c9" containerName="main" containerID="cri-o://1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd" gracePeriod=45
Aug 04 10:02:05.829325 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.829306    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:02:05.862103 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862078    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-model-cache\") pod \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") "
Aug 04 10:02:05.862254 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862171    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-home\") pod \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") "
Aug 04 10:02:05.862254 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862203    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tls-certs\") pod \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") "
Aug 04 10:02:05.862371 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862254    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-dshm\") pod \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") "
Aug 04 10:02:05.862371 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862295    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tmp-dir\") pod \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") "
Aug 04 10:02:05.862371 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862329    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ac9e57f-1a60-4a28-8b76-064ae871b6c9" (UID: "2ac9e57f-1a60-4a28-8b76-064ae871b6c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:02:05.862371 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862348    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh4lb\" (UniqueName: \"kubernetes.io/projected/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-kube-api-access-kh4lb\") pod \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\" (UID: \"2ac9e57f-1a60-4a28-8b76-064ae871b6c9\") "
Aug 04 10:02:05.862554 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862435    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-home" (OuterVolumeSpecName: "home") pod "2ac9e57f-1a60-4a28-8b76-064ae871b6c9" (UID: "2ac9e57f-1a60-4a28-8b76-064ae871b6c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:02:05.862601 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862560    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ac9e57f-1a60-4a28-8b76-064ae871b6c9" (UID: "2ac9e57f-1a60-4a28-8b76-064ae871b6c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:02:05.862660 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862644    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-home\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:02:05.862697 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862680    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tmp-dir\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:02:05.862697 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.862691    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-model-cache\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:02:05.864533 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.864505    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ac9e57f-1a60-4a28-8b76-064ae871b6c9" (UID: "2ac9e57f-1a60-4a28-8b76-064ae871b6c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 10:02:05.864627 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.864540    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-dshm" (OuterVolumeSpecName: "dshm") pod "2ac9e57f-1a60-4a28-8b76-064ae871b6c9" (UID: "2ac9e57f-1a60-4a28-8b76-064ae871b6c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:02:05.864627 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.864547    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-kube-api-access-kh4lb" (OuterVolumeSpecName: "kube-api-access-kh4lb") pod "2ac9e57f-1a60-4a28-8b76-064ae871b6c9" (UID: "2ac9e57f-1a60-4a28-8b76-064ae871b6c9"). InnerVolumeSpecName "kube-api-access-kh4lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:02:05.964040 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.964007    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh4lb\" (UniqueName: \"kubernetes.io/projected/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-kube-api-access-kh4lb\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:02:05.964040 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.964034    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-tls-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:02:05.964040 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:05.964045    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ac9e57f-1a60-4a28-8b76-064ae871b6c9-dshm\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:02:06.214844 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.214727    2558 generic.go:356] "Generic (PLEG): container finished" podID="2ac9e57f-1a60-4a28-8b76-064ae871b6c9" containerID="1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd" exitCode=0
Aug 04 10:02:06.214844 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.214815    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"
Aug 04 10:02:06.215031 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.214828    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds" event={"ID":"2ac9e57f-1a60-4a28-8b76-064ae871b6c9","Type":"ContainerDied","Data":"1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd"}
Aug 04 10:02:06.215031 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.214889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds" event={"ID":"2ac9e57f-1a60-4a28-8b76-064ae871b6c9","Type":"ContainerDied","Data":"db299f7604a25e4b3c07454958121690aa37432c6976f0fa664d6887ed7a09e3"}
Aug 04 10:02:06.215031 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.214909    2558 scope.go:117] "RemoveContainer" containerID="1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd"
Aug 04 10:02:06.224133 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.224115    2558 scope.go:117] "RemoveContainer" containerID="1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd"
Aug 04 10:02:06.224396 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:02:06.224376    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd\": container with ID starting with 1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd not found: ID does not exist" containerID="1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd"
Aug 04 10:02:06.224447 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.224404    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd"} err="failed to get container status \"1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd\": rpc error: code = NotFound desc = could not find container \"1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd\": container with ID starting with 1cba8d505dc6b5b6d2108b229f0c64b68d5f81b108f7d8a9bfe1b85dc784b6fd not found: ID does not exist"
Aug 04 10:02:06.246676 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.246649    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"]
Aug 04 10:02:06.252615 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:06.252595    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b0bc81/rate-test-model-b0bc81-kserve-7d994f77f5-gjqds"]
Aug 04 10:02:07.295456 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:07.295423    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ac9e57f-1a60-4a28-8b76-064ae871b6c9" path="/var/lib/kubelet/pods/2ac9e57f-1a60-4a28-8b76-064ae871b6c9/volumes"
Aug 04 10:02:15.067308 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.067261    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"]
Aug 04 10:02:15.067879 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.067854    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac9e57f-1a60-4a28-8b76-064ae871b6c9" containerName="main"
Aug 04 10:02:15.067879 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.067880    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac9e57f-1a60-4a28-8b76-064ae871b6c9" containerName="main"
Aug 04 10:02:15.068050 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.067951    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac9e57f-1a60-4a28-8b76-064ae871b6c9" containerName="main"
Aug 04 10:02:15.073586 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.073559    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.077521 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.077499    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-fc7b93-dockercfg-mtgdz\""
Aug 04 10:02:15.081683 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.081663    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-fc7b93\""
Aug 04 10:02:15.093417 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.093374    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"]
Aug 04 10:02:15.147137 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.147098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhpmf\" (UniqueName: \"kubernetes.io/projected/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-kube-api-access-fhpmf\") pod \"payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.147311 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.147208    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.248214 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.248176    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhpmf\" (UniqueName: \"kubernetes.io/projected/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-kube-api-access-fhpmf\") pod \"payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.248481 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.248244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.249009 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.248986    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.258282 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.258257    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhpmf\" (UniqueName: \"kubernetes.io/projected/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-kube-api-access-fhpmf\") pod \"payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.384814 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.384775    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:15.539033 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:15.539007    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"]
Aug 04 10:02:15.541011 ip-10-0-143-106 kubenswrapper[2558]: W0804 10:02:15.540972    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56bee78_5ae6_41c1_a6c1_71b8cdd83d22.slice/crio-93bf43d440dd344919c8188e61f3348923fe50511029c04fb7b8a1628027b9c0 WatchSource:0}: Error finding container 93bf43d440dd344919c8188e61f3348923fe50511029c04fb7b8a1628027b9c0: Status 404 returned error can't find the container with id 93bf43d440dd344919c8188e61f3348923fe50511029c04fb7b8a1628027b9c0
Aug 04 10:02:16.271049 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:16.270921    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587" event={"ID":"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22","Type":"ContainerStarted","Data":"6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43"}
Aug 04 10:02:16.271049 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:16.270991    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:16.271049 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:16.271016    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587" event={"ID":"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22","Type":"ContainerStarted","Data":"93bf43d440dd344919c8188e61f3348923fe50511029c04fb7b8a1628027b9c0"}
Aug 04 10:02:16.295071 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:16.295021    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587" podStartSLOduration=1.295004619 podStartE2EDuration="1.295004619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:02:16.293509291 +0000 UTC m=+3267.551466345" watchObservedRunningTime="2026-08-04 10:02:16.295004619 +0000 UTC m=+3267.552961673"
Aug 04 10:02:17.165435 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:17.165401    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:02:27.278492 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:27.278460    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:02:39.153354 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.153323    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"]
Aug 04 10:02:39.156992 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.156970    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.159715 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.159695    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-90d908-openshift-default-dockercfg-pxsmn\""
Aug 04 10:02:39.179051 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.179023    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"]
Aug 04 10:02:39.270186 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270153    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6hvq\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-kube-api-access-g6hvq\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270186 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270186    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-data\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270416 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270225    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-podinfo\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270416 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270290    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istiod-ca-cert\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270416 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270367    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-credential-socket\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270416 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270407    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-socket\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270619 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270441    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-envoy\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270619 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270460    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-certs\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.270619 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.270515    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-token\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.371578 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371544    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-podinfo\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.371814 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istiod-ca-cert\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.371814 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371635    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-credential-socket\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.371814 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371661    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-socket\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.371814 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-envoy\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.371814 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-certs\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.371814 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371766    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-token\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.372147 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371849    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6hvq\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-kube-api-access-g6hvq\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.372147 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.371876    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-data\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.372353 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.372314    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-data\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.372594 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.372564    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-certs\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.372894 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.372840    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-credential-socket\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.373007 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.372968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istiod-ca-cert\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.373131 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.373105    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-socket\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.374956 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.374935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-envoy\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.375122 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.375102    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-podinfo\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.384957 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.384936    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-token\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.385200 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.385182    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6hvq\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-kube-api-access-g6hvq\") pod \"e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") " pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.467865 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.467805    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:39.622438 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.622406    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"]
Aug 04 10:02:39.626683 ip-10-0-143-106 kubenswrapper[2558]: W0804 10:02:39.626657    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b5f11c_4f3e_4475_bc21_6f4d489cee00.slice/crio-e925f2be5bf4de53dfa1909cb0e64f2d138e45de6de0f40ba5592c2d86f874b4 WatchSource:0}: Error finding container e925f2be5bf4de53dfa1909cb0e64f2d138e45de6de0f40ba5592c2d86f874b4: Status 404 returned error can't find the container with id e925f2be5bf4de53dfa1909cb0e64f2d138e45de6de0f40ba5592c2d86f874b4
Aug 04 10:02:39.628810 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.628780    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 10:02:39.628874 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.628845    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 10:02:39.628915 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:39.628873    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 10:02:40.389256 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:40.389217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" event={"ID":"18b5f11c-4f3e-4475-bc21-6f4d489cee00","Type":"ContainerStarted","Data":"53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872"}
Aug 04 10:02:40.389658 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:40.389272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" event={"ID":"18b5f11c-4f3e-4475-bc21-6f4d489cee00","Type":"ContainerStarted","Data":"e925f2be5bf4de53dfa1909cb0e64f2d138e45de6de0f40ba5592c2d86f874b4"}
Aug 04 10:02:40.426831 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:40.426782    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podStartSLOduration=1.426766891 podStartE2EDuration="1.426766891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:02:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:02:40.424760201 +0000 UTC m=+3291.682717246" watchObservedRunningTime="2026-08-04 10:02:40.426766891 +0000 UTC m=+3291.684723946"
Aug 04 10:02:40.468096 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:40.468069    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:40.473023 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:40.473002    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:41.393491 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:41.393461    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:41.394488 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:41.394464    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:46.850545 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:46.850511    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:02:56.545375 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:56.544474    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:56296->10.133.0.65:15021: read: connection reset by peer" start-of-body=
Aug 04 10:02:56.545375 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:56.544558    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:56296->10.133.0.65:15021: read: connection reset by peer"
Aug 04 10:02:57.491748 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:57.491697    2558 generic.go:356] "Generic (PLEG): container finished" podID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerID="53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872" exitCode=0
Aug 04 10:02:57.491976 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:57.491775    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" event={"ID":"18b5f11c-4f3e-4475-bc21-6f4d489cee00","Type":"ContainerDied","Data":"53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872"}
Aug 04 10:02:57.492214 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:57.492197    2558 scope.go:117] "RemoveContainer" containerID="53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872"
Aug 04 10:02:57.493259 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:57.493235    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 10:02:57.493319 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:57.493301    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 10:02:57.493356 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:57.493337    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 10:02:58.498518 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:58.498483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" event={"ID":"18b5f11c-4f3e-4475-bc21-6f4d489cee00","Type":"ContainerStarted","Data":"97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8"}
Aug 04 10:02:59.468248 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:59.468217    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:02:59.468490 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:59.468433    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Aug 04 10:02:59.468568 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:02:59.468490    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Aug 04 10:03:00.469186 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:00.469149    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Aug 04 10:03:00.469567 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:00.469206    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Aug 04 10:03:01.468341 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:01.468306    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Aug 04 10:03:01.468517 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:01.468359    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Aug 04 10:03:02.468485 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:02.468445    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Aug 04 10:03:02.468900 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:02.468516    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Aug 04 10:03:03.468954 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:03.468912    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Aug 04 10:03:03.469351 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:03.468978    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Aug 04 10:03:04.472181 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:04.472149    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:03:04.472645 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:04.472530    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:03:04.473273 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:04.473256    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:03:14.464985 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:14.464911    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-kbzjm_d21cb922-e090-4055-b1a7-371acf923f6d/payload-processing/0.log"
Aug 04 10:03:19.165558 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:03:19.165519    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:04:03.753034 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:03.752990    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:04:15.057483 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:15.057450    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587_a56bee78-5ae6-41c1-a6c1-71b8cdd83d22/payload-processing/0.log"
Aug 04 10:04:15.181375 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:15.181349    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-kbzjm_d21cb922-e090-4055-b1a7-371acf923f6d/payload-processing/0.log"
Aug 04 10:04:21.039587 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.039551    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"]
Aug 04 10:04:21.043143 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.043124    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.045983 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.045964    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-e0d739-dockercfg-nlxzn\""
Aug 04 10:04:21.046073 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.046055    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-e0d739\""
Aug 04 10:04:21.058219 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.058192    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"]
Aug 04 10:04:21.110802 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.110764    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"]
Aug 04 10:04:21.119161 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.119130    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.131314 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.131280    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"]
Aug 04 10:04:21.178858 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.178820    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dwm6\" (UniqueName: \"kubernetes.io/projected/67bebbe7-6e56-4b8c-b850-f5adc302d422-kube-api-access-4dwm6\") pod \"payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.179020 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.178963    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67bebbe7-6e56-4b8c-b850-f5adc302d422-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.279599 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.279563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67bebbe7-6e56-4b8c-b850-f5adc302d422-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.279787 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.279638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f8514-87ec-4089-a4d8-ddf9605cf750-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.279787 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.279658    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lffr9\" (UniqueName: \"kubernetes.io/projected/d38f8514-87ec-4089-a4d8-ddf9605cf750-kube-api-access-lffr9\") pod \"payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.279787 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.279715    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dwm6\" (UniqueName: \"kubernetes.io/projected/67bebbe7-6e56-4b8c-b850-f5adc302d422-kube-api-access-4dwm6\") pod \"payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.280459 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.280434    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67bebbe7-6e56-4b8c-b850-f5adc302d422-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.288854 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.288825    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dwm6\" (UniqueName: \"kubernetes.io/projected/67bebbe7-6e56-4b8c-b850-f5adc302d422-kube-api-access-4dwm6\") pod \"payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.354303 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.354276    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.380879 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.380851    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f8514-87ec-4089-a4d8-ddf9605cf750-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.381032 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.380894    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lffr9\" (UniqueName: \"kubernetes.io/projected/d38f8514-87ec-4089-a4d8-ddf9605cf750-kube-api-access-lffr9\") pod \"payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.381566 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.381544    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f8514-87ec-4089-a4d8-ddf9605cf750-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.398540 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.398232    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lffr9\" (UniqueName: \"kubernetes.io/projected/d38f8514-87ec-4089-a4d8-ddf9605cf750-kube-api-access-lffr9\") pod \"payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.446010 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.444992    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.521381 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.521007    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"]
Aug 04 10:04:21.603758 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.603717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"]
Aug 04 10:04:21.605484 ip-10-0-143-106 kubenswrapper[2558]: W0804 10:04:21.605415    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38f8514_87ec_4089_a4d8_ddf9605cf750.slice/crio-5d720e20cc9fc70d800b69caf929ce5d4344b96ed5922f4351a32d172ab79193 WatchSource:0}: Error finding container 5d720e20cc9fc70d800b69caf929ce5d4344b96ed5922f4351a32d172ab79193: Status 404 returned error can't find the container with id 5d720e20cc9fc70d800b69caf929ce5d4344b96ed5922f4351a32d172ab79193
Aug 04 10:04:21.903832 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.902793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg" event={"ID":"67bebbe7-6e56-4b8c-b850-f5adc302d422","Type":"ContainerStarted","Data":"bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7"}
Aug 04 10:04:21.903832 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.902863    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg" event={"ID":"67bebbe7-6e56-4b8c-b850-f5adc302d422","Type":"ContainerStarted","Data":"190e866df6bb59549b4b87edd1cd71c6c304e28e66bf35f130ecd4376b1d080b"}
Aug 04 10:04:21.903832 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.903692    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:21.906333 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.905897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2" event={"ID":"d38f8514-87ec-4089-a4d8-ddf9605cf750","Type":"ContainerStarted","Data":"c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566"}
Aug 04 10:04:21.906333 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.905949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2" event={"ID":"d38f8514-87ec-4089-a4d8-ddf9605cf750","Type":"ContainerStarted","Data":"5d720e20cc9fc70d800b69caf929ce5d4344b96ed5922f4351a32d172ab79193"}
Aug 04 10:04:21.906333 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.906059    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:21.963055 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.962989    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg" podStartSLOduration=0.962968432 podStartE2EDuration="962.968432ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:04:21.932427738 +0000 UTC m=+3393.190384791" watchObservedRunningTime="2026-08-04 10:04:21.962968432 +0000 UTC m=+3393.220925487"
Aug 04 10:04:21.963710 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:21.963683    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2" podStartSLOduration=0.96367479 podStartE2EDuration="963.67479ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:04:21.96099642 +0000 UTC m=+3393.218953474" watchObservedRunningTime="2026-08-04 10:04:21.96367479 +0000 UTC m=+3393.221631847"
Aug 04 10:04:25.643868 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:25.643839    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:04:32.912932 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:32.912900    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:33.918943 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:33.918914    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:50.589366 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.589331    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"]
Aug 04 10:04:50.589758 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.589596    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2" podUID="d38f8514-87ec-4089-a4d8-ddf9605cf750" containerName="payload-processing" containerID="cri-o://c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566" gracePeriod=30
Aug 04 10:04:50.600167 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.600134    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"]
Aug 04 10:04:50.600385 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.600352    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg" podUID="67bebbe7-6e56-4b8c-b850-f5adc302d422" containerName="payload-pre-processing" containerID="cri-o://bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7" gracePeriod=30
Aug 04 10:04:50.862529 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.862503    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:50.898380 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.898360    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:50.959721 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.959692    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f8514-87ec-4089-a4d8-ddf9605cf750-plugins-config-volume\") pod \"d38f8514-87ec-4089-a4d8-ddf9605cf750\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") "
Aug 04 10:04:50.959890 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.959747    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lffr9\" (UniqueName: \"kubernetes.io/projected/d38f8514-87ec-4089-a4d8-ddf9605cf750-kube-api-access-lffr9\") pod \"d38f8514-87ec-4089-a4d8-ddf9605cf750\" (UID: \"d38f8514-87ec-4089-a4d8-ddf9605cf750\") "
Aug 04 10:04:50.960160 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.960134    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38f8514-87ec-4089-a4d8-ddf9605cf750-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d38f8514-87ec-4089-a4d8-ddf9605cf750" (UID: "d38f8514-87ec-4089-a4d8-ddf9605cf750"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:04:50.961946 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:50.961928    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38f8514-87ec-4089-a4d8-ddf9605cf750-kube-api-access-lffr9" (OuterVolumeSpecName: "kube-api-access-lffr9") pod "d38f8514-87ec-4089-a4d8-ddf9605cf750" (UID: "d38f8514-87ec-4089-a4d8-ddf9605cf750"). InnerVolumeSpecName "kube-api-access-lffr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:04:51.060265 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060239    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dwm6\" (UniqueName: \"kubernetes.io/projected/67bebbe7-6e56-4b8c-b850-f5adc302d422-kube-api-access-4dwm6\") pod \"67bebbe7-6e56-4b8c-b850-f5adc302d422\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") "
Aug 04 10:04:51.060493 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060293    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67bebbe7-6e56-4b8c-b850-f5adc302d422-plugins-config-volume\") pod \"67bebbe7-6e56-4b8c-b850-f5adc302d422\" (UID: \"67bebbe7-6e56-4b8c-b850-f5adc302d422\") "
Aug 04 10:04:51.060547 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060482    2558 generic.go:356] "Generic (PLEG): container finished" podID="d38f8514-87ec-4089-a4d8-ddf9605cf750" containerID="c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566" exitCode=0
Aug 04 10:04:51.060588 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060541    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"
Aug 04 10:04:51.060640 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2" event={"ID":"d38f8514-87ec-4089-a4d8-ddf9605cf750","Type":"ContainerDied","Data":"c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566"}
Aug 04 10:04:51.060640 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060594    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f8514-87ec-4089-a4d8-ddf9605cf750-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:04:51.060640 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060615    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lffr9\" (UniqueName: \"kubernetes.io/projected/d38f8514-87ec-4089-a4d8-ddf9605cf750-kube-api-access-lffr9\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:04:51.060813 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2" event={"ID":"d38f8514-87ec-4089-a4d8-ddf9605cf750","Type":"ContainerDied","Data":"5d720e20cc9fc70d800b69caf929ce5d4344b96ed5922f4351a32d172ab79193"}
Aug 04 10:04:51.060813 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060714    2558 scope.go:117] "RemoveContainer" containerID="c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566"
Aug 04 10:04:51.060813 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.060763    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67bebbe7-6e56-4b8c-b850-f5adc302d422-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67bebbe7-6e56-4b8c-b850-f5adc302d422" (UID: "67bebbe7-6e56-4b8c-b850-f5adc302d422"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:04:51.062106 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.062085    2558 generic.go:356] "Generic (PLEG): container finished" podID="67bebbe7-6e56-4b8c-b850-f5adc302d422" containerID="bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7" exitCode=0
Aug 04 10:04:51.062198 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.062147    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"
Aug 04 10:04:51.062258 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.062196    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg" event={"ID":"67bebbe7-6e56-4b8c-b850-f5adc302d422","Type":"ContainerDied","Data":"bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7"}
Aug 04 10:04:51.062258 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.062241    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg" event={"ID":"67bebbe7-6e56-4b8c-b850-f5adc302d422","Type":"ContainerDied","Data":"190e866df6bb59549b4b87edd1cd71c6c304e28e66bf35f130ecd4376b1d080b"}
Aug 04 10:04:51.062847 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.062827    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67bebbe7-6e56-4b8c-b850-f5adc302d422-kube-api-access-4dwm6" (OuterVolumeSpecName: "kube-api-access-4dwm6") pod "67bebbe7-6e56-4b8c-b850-f5adc302d422" (UID: "67bebbe7-6e56-4b8c-b850-f5adc302d422"). InnerVolumeSpecName "kube-api-access-4dwm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:04:51.070923 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.070906    2558 scope.go:117] "RemoveContainer" containerID="c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566"
Aug 04 10:04:51.071189 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:04:51.071169    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566\": container with ID starting with c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566 not found: ID does not exist" containerID="c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566"
Aug 04 10:04:51.071244 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.071197    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566"} err="failed to get container status \"c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566\": rpc error: code = NotFound desc = could not find container \"c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566\": container with ID starting with c0e3e883578015fccb1a044c98b221e0237f2d2e67aa637e87d99ac1b93ab566 not found: ID does not exist"
Aug 04 10:04:51.071244 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.071213    2558 scope.go:117] "RemoveContainer" containerID="bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7"
Aug 04 10:04:51.078995 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.078975    2558 scope.go:117] "RemoveContainer" containerID="bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7"
Aug 04 10:04:51.079230 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:04:51.079213    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7\": container with ID starting with bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7 not found: ID does not exist" containerID="bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7"
Aug 04 10:04:51.079276 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.079237    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7"} err="failed to get container status \"bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7\": rpc error: code = NotFound desc = could not find container \"bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7\": container with ID starting with bfcf8c09f55dd22f799b135f4d10bbf8cd808a5d31b006e2257af0a97049d0b7 not found: ID does not exist"
Aug 04 10:04:51.084535 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.084491    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"]
Aug 04 10:04:51.087134 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.087112    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e0d739-58c6849564-8qhv2"]
Aug 04 10:04:51.161498 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.161434    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dwm6\" (UniqueName: \"kubernetes.io/projected/67bebbe7-6e56-4b8c-b850-f5adc302d422-kube-api-access-4dwm6\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:04:51.161498 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.161455    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67bebbe7-6e56-4b8c-b850-f5adc302d422-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:04:51.295941 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.295912    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38f8514-87ec-4089-a4d8-ddf9605cf750" path="/var/lib/kubelet/pods/d38f8514-87ec-4089-a4d8-ddf9605cf750/volumes"
Aug 04 10:04:51.378069 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.378042    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"]
Aug 04 10:04:51.382977 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:51.382955    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e0d739-56b596f5b8-qz6zg"]
Aug 04 10:04:53.295423 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:53.295387    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67bebbe7-6e56-4b8c-b850-f5adc302d422" path="/var/lib/kubelet/pods/67bebbe7-6e56-4b8c-b850-f5adc302d422/volumes"
Aug 04 10:04:53.856485 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:04:53.856451    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:05:14.442858 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:14.442821    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:05:16.733687 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:16.733654    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"]
Aug 04 10:05:16.734215 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:16.733962    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587" podUID="a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" containerName="payload-processing" containerID="cri-o://6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43" gracePeriod=30
Aug 04 10:05:16.993193 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:16.993139    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:05:17.082173 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.082143    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-plugins-config-volume\") pod \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") "
Aug 04 10:05:17.082335 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.082241    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhpmf\" (UniqueName: \"kubernetes.io/projected/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-kube-api-access-fhpmf\") pod \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\" (UID: \"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22\") "
Aug 04 10:05:17.082506 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.082483    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" (UID: "a56bee78-5ae6-41c1-a6c1-71b8cdd83d22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:05:17.084424 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.084393    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-kube-api-access-fhpmf" (OuterVolumeSpecName: "kube-api-access-fhpmf") pod "a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" (UID: "a56bee78-5ae6-41c1-a6c1-71b8cdd83d22"). InnerVolumeSpecName "kube-api-access-fhpmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:05:17.183321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.183284    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhpmf\" (UniqueName: \"kubernetes.io/projected/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-kube-api-access-fhpmf\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:05:17.183321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.183316    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:05:17.191061 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.191033    2558 generic.go:356] "Generic (PLEG): container finished" podID="a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" containerID="6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43" exitCode=0
Aug 04 10:05:17.191163 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.191079    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"
Aug 04 10:05:17.191163 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.191111    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587" event={"ID":"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22","Type":"ContainerDied","Data":"6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43"}
Aug 04 10:05:17.191163 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.191159    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587" event={"ID":"a56bee78-5ae6-41c1-a6c1-71b8cdd83d22","Type":"ContainerDied","Data":"93bf43d440dd344919c8188e61f3348923fe50511029c04fb7b8a1628027b9c0"}
Aug 04 10:05:17.191280 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.191183    2558 scope.go:117] "RemoveContainer" containerID="6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43"
Aug 04 10:05:17.200016 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.200000    2558 scope.go:117] "RemoveContainer" containerID="6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43"
Aug 04 10:05:17.200267 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:05:17.200247    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43\": container with ID starting with 6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43 not found: ID does not exist" containerID="6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43"
Aug 04 10:05:17.200329 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.200275    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43"} err="failed to get container status \"6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43\": rpc error: code = NotFound desc = could not find container \"6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43\": container with ID starting with 6727ae16bea02d8b921e2c52126c2e3efc47270594b97930d15be37fa7da2f43 not found: ID does not exist"
Aug 04 10:05:17.232235 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.232211    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"]
Aug 04 10:05:17.265843 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.265788    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fc7b93-5896995db6-x5587"]
Aug 04 10:05:17.295331 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:17.295305    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" path="/var/lib/kubelet/pods/a56bee78-5ae6-41c1-a6c1-71b8cdd83d22/volumes"
Aug 04 10:05:23.355101 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:05:23.355060    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:06:07.371115 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371081    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"]
Aug 04 10:06:07.371537 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371434    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67bebbe7-6e56-4b8c-b850-f5adc302d422" containerName="payload-pre-processing"
Aug 04 10:06:07.371537 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371445    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="67bebbe7-6e56-4b8c-b850-f5adc302d422" containerName="payload-pre-processing"
Aug 04 10:06:07.371537 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371464    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" containerName="payload-processing"
Aug 04 10:06:07.371537 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371470    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" containerName="payload-processing"
Aug 04 10:06:07.371537 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371484    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38f8514-87ec-4089-a4d8-ddf9605cf750" containerName="payload-processing"
Aug 04 10:06:07.371537 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371489    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38f8514-87ec-4089-a4d8-ddf9605cf750" containerName="payload-processing"
Aug 04 10:06:07.371726 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371545    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a56bee78-5ae6-41c1-a6c1-71b8cdd83d22" containerName="payload-processing"
Aug 04 10:06:07.371726 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371555    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="67bebbe7-6e56-4b8c-b850-f5adc302d422" containerName="payload-pre-processing"
Aug 04 10:06:07.371726 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.371563    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38f8514-87ec-4089-a4d8-ddf9605cf750" containerName="payload-processing"
Aug 04 10:06:07.374679 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.374661    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"
Aug 04 10:06:07.377062 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.377043    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\""
Aug 04 10:06:07.384516 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.384490    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"]
Aug 04 10:06:07.420035 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.420000    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3589357-c163-4c2f-9181-0e603b5229e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l\" (UID: \"d3589357-c163-4c2f-9181-0e603b5229e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"
Aug 04 10:06:07.521349 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.521321    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3589357-c163-4c2f-9181-0e603b5229e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l\" (UID: \"d3589357-c163-4c2f-9181-0e603b5229e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"
Aug 04 10:06:07.521957 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.521937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3589357-c163-4c2f-9181-0e603b5229e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l\" (UID: \"d3589357-c163-4c2f-9181-0e603b5229e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"
Aug 04 10:06:07.686514 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.686436    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"
Aug 04 10:06:07.837470 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:07.837443    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"]
Aug 04 10:06:07.838479 ip-10-0-143-106 kubenswrapper[2558]: W0804 10:06:07.838455    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3589357_c163_4c2f_9181_0e603b5229e8.slice/crio-547165ade7900be5e4f9ce0d13637319992d7090bb469d914c0437eefc1fd027 WatchSource:0}: Error finding container 547165ade7900be5e4f9ce0d13637319992d7090bb469d914c0437eefc1fd027: Status 404 returned error can't find the container with id 547165ade7900be5e4f9ce0d13637319992d7090bb469d914c0437eefc1fd027
Aug 04 10:06:08.433457 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:08.433409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l" event={"ID":"d3589357-c163-4c2f-9181-0e603b5229e8","Type":"ContainerStarted","Data":"bd5b84c639f37eabb9f9970b82c8ac5ca3d94b56901cad67b860adf60ca47777"}
Aug 04 10:06:08.433892 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:08.433483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l" event={"ID":"d3589357-c163-4c2f-9181-0e603b5229e8","Type":"ContainerStarted","Data":"547165ade7900be5e4f9ce0d13637319992d7090bb469d914c0437eefc1fd027"}
Aug 04 10:06:08.457634 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:08.457589    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l" podStartSLOduration=1.457574511 podStartE2EDuration="1.457574511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:06:08.455948115 +0000 UTC m=+3499.713905182" watchObservedRunningTime="2026-08-04 10:06:08.457574511 +0000 UTC m=+3499.715531642"
Aug 04 10:06:09.439453 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:09.439411    2558 generic.go:356] "Generic (PLEG): container finished" podID="d3589357-c163-4c2f-9181-0e603b5229e8" containerID="bd5b84c639f37eabb9f9970b82c8ac5ca3d94b56901cad67b860adf60ca47777" exitCode=0
Aug 04 10:06:09.439829 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:09.439497    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l" event={"ID":"d3589357-c163-4c2f-9181-0e603b5229e8","Type":"ContainerDied","Data":"bd5b84c639f37eabb9f9970b82c8ac5ca3d94b56901cad67b860adf60ca47777"}
Aug 04 10:06:10.578837 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:10.578809    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"
Aug 04 10:06:10.648087 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:10.648060    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3589357-c163-4c2f-9181-0e603b5229e8-maas-api-service-ca\") pod \"d3589357-c163-4c2f-9181-0e603b5229e8\" (UID: \"d3589357-c163-4c2f-9181-0e603b5229e8\") "
Aug 04 10:06:10.648396 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:10.648375    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3589357-c163-4c2f-9181-0e603b5229e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d3589357-c163-4c2f-9181-0e603b5229e8" (UID: "d3589357-c163-4c2f-9181-0e603b5229e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:06:10.749880 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:10.749807    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3589357-c163-4c2f-9181-0e603b5229e8-maas-api-service-ca\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:11.451018 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:11.450991    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"
Aug 04 10:06:11.451186 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:11.451033    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l" event={"ID":"d3589357-c163-4c2f-9181-0e603b5229e8","Type":"ContainerDied","Data":"547165ade7900be5e4f9ce0d13637319992d7090bb469d914c0437eefc1fd027"}
Aug 04 10:06:11.451186 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:11.451077    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="547165ade7900be5e4f9ce0d13637319992d7090bb469d914c0437eefc1fd027"
Aug 04 10:06:18.174073 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:18.174018    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"]
Aug 04 10:06:18.179197 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:18.179099    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-90d908-tfq4l"]
Aug 04 10:06:19.295899 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:19.295867    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3589357-c163-4c2f-9181-0e603b5229e8" path="/var/lib/kubelet/pods/d3589357-c163-4c2f-9181-0e603b5229e8/volumes"
Aug 04 10:06:20.338439 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:20.338396    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:06:24.707538 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:24.707505    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"]
Aug 04 10:06:24.707937 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:24.707724    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy" containerID="cri-o://97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8" gracePeriod=30
Aug 04 10:06:29.978103 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:29.978036    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:06:30.128039 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128012    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istiod-ca-cert\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128236 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128052    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-token\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128236 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128072    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-data\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128236 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128100    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-credential-socket\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128236 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128142    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-certs\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128236 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128171    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-envoy\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128236 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128197    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-socket\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128271    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6hvq\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-kube-api-access-g6hvq\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128320    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-podinfo\") pod \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\" (UID: \"18b5f11c-4f3e-4475-bc21-6f4d489cee00\") "
Aug 04 10:06:30.128548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128422    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-data" (OuterVolumeSpecName: "istio-data") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:06:30.128548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128507    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:06:30.128548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128517    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:06:30.128548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128526    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:06:30.128548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128545    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:06:30.128906 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128659    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.128906 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128671    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-workload-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.128906 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128680    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istiod-ca-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.128906 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128688    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-data\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.128906 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.128696    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-credential-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.130395 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.130371    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-token" (OuterVolumeSpecName: "istio-token") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:06:30.130560 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.130545    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 10:06:30.131131 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.131108    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-kube-api-access-g6hvq" (OuterVolumeSpecName: "kube-api-access-g6hvq") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "kube-api-access-g6hvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:06:30.131204 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.131121    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18b5f11c-4f3e-4475-bc21-6f4d489cee00" (UID: "18b5f11c-4f3e-4475-bc21-6f4d489cee00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:06:30.229769 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.229688    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-token\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.229769 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.229713    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-envoy\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.229769 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.229723    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6hvq\" (UniqueName: \"kubernetes.io/projected/18b5f11c-4f3e-4475-bc21-6f4d489cee00-kube-api-access-g6hvq\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.229769 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.229752    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18b5f11c-4f3e-4475-bc21-6f4d489cee00-istio-podinfo\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:30.545144 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.545050    2558 generic.go:356] "Generic (PLEG): container finished" podID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerID="97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8" exitCode=0
Aug 04 10:06:30.545144 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.545100    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"
Aug 04 10:06:30.545362 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.545140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" event={"ID":"18b5f11c-4f3e-4475-bc21-6f4d489cee00","Type":"ContainerDied","Data":"97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8"}
Aug 04 10:06:30.545362 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.545194    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb" event={"ID":"18b5f11c-4f3e-4475-bc21-6f4d489cee00","Type":"ContainerDied","Data":"e925f2be5bf4de53dfa1909cb0e64f2d138e45de6de0f40ba5592c2d86f874b4"}
Aug 04 10:06:30.545362 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.545217    2558 scope.go:117] "RemoveContainer" containerID="97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8"
Aug 04 10:06:30.554337 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.554321    2558 scope.go:117] "RemoveContainer" containerID="53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872"
Aug 04 10:06:30.562427 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.562411    2558 scope.go:117] "RemoveContainer" containerID="97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8"
Aug 04 10:06:30.562694 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:06:30.562677    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8\": container with ID starting with 97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8 not found: ID does not exist" containerID="97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8"
Aug 04 10:06:30.562758 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.562701    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8"} err="failed to get container status \"97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8\": rpc error: code = NotFound desc = could not find container \"97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8\": container with ID starting with 97edeeb140644d2597ee6d037233a0e6694ededb57f64b0cb24ad6a2587ff0a8 not found: ID does not exist"
Aug 04 10:06:30.562758 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.562720    2558 scope.go:117] "RemoveContainer" containerID="53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872"
Aug 04 10:06:30.562969 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:06:30.562950    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872\": container with ID starting with 53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872 not found: ID does not exist" containerID="53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872"
Aug 04 10:06:30.563006 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.562976    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872"} err="failed to get container status \"53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872\": rpc error: code = NotFound desc = could not find container \"53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872\": container with ID starting with 53a10c699fe8c01450f22b3305f878fc639ce97cbfcf47195b33472e42164872 not found: ID does not exist"
Aug 04 10:06:30.567876 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.567853    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"]
Aug 04 10:06:30.573046 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:30.573022    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-90d908-openshift-default-6d658f5688-k7rtb"]
Aug 04 10:06:31.296682 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:31.296646    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" path="/var/lib/kubelet/pods/18b5f11c-4f3e-4475-bc21-6f4d489cee00/volumes"
Aug 04 10:06:35.785148 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:35.785111    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"]
Aug 04 10:06:35.785576 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:35.785327    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf" podUID="a78d9c91-20d3-4b2b-b9de-1e522923266f" containerName="payload-pre-processing" containerID="cri-o://a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2" gracePeriod=30
Aug 04 10:06:36.050475 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.050448    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 10:06:36.180674 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.180645    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a78d9c91-20d3-4b2b-b9de-1e522923266f-plugins-config-volume\") pod \"a78d9c91-20d3-4b2b-b9de-1e522923266f\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") "
Aug 04 10:06:36.180874 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.180763    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7qgg\" (UniqueName: \"kubernetes.io/projected/a78d9c91-20d3-4b2b-b9de-1e522923266f-kube-api-access-c7qgg\") pod \"a78d9c91-20d3-4b2b-b9de-1e522923266f\" (UID: \"a78d9c91-20d3-4b2b-b9de-1e522923266f\") "
Aug 04 10:06:36.181055 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.181033    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78d9c91-20d3-4b2b-b9de-1e522923266f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a78d9c91-20d3-4b2b-b9de-1e522923266f" (UID: "a78d9c91-20d3-4b2b-b9de-1e522923266f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:06:36.182923 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.182900    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a78d9c91-20d3-4b2b-b9de-1e522923266f-kube-api-access-c7qgg" (OuterVolumeSpecName: "kube-api-access-c7qgg") pod "a78d9c91-20d3-4b2b-b9de-1e522923266f" (UID: "a78d9c91-20d3-4b2b-b9de-1e522923266f"). InnerVolumeSpecName "kube-api-access-c7qgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:06:36.281819 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.281783    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7qgg\" (UniqueName: \"kubernetes.io/projected/a78d9c91-20d3-4b2b-b9de-1e522923266f-kube-api-access-c7qgg\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:36.281819 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.281814    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a78d9c91-20d3-4b2b-b9de-1e522923266f-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:36.576316 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.576282    2558 generic.go:356] "Generic (PLEG): container finished" podID="a78d9c91-20d3-4b2b-b9de-1e522923266f" containerID="a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2" exitCode=0
Aug 04 10:06:36.576484 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.576322    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"
Aug 04 10:06:36.576484 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.576339    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf" event={"ID":"a78d9c91-20d3-4b2b-b9de-1e522923266f","Type":"ContainerDied","Data":"a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2"}
Aug 04 10:06:36.576484 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.576380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf" event={"ID":"a78d9c91-20d3-4b2b-b9de-1e522923266f","Type":"ContainerDied","Data":"ae0d82827c5ebe47d4f1abfce2496dcb4f8356724bbfe6c24bef8c6716681133"}
Aug 04 10:06:36.576484 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.576407    2558 scope.go:117] "RemoveContainer" containerID="a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2"
Aug 04 10:06:36.585363 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.585343    2558 scope.go:117] "RemoveContainer" containerID="a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2"
Aug 04 10:06:36.585613 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:06:36.585595    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2\": container with ID starting with a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2 not found: ID does not exist" containerID="a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2"
Aug 04 10:06:36.585666 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.585621    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2"} err="failed to get container status \"a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2\": rpc error: code = NotFound desc = could not find container \"a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2\": container with ID starting with a1506caa856da532ae7d799d6daf86d685a0c6916ab759ce6cda2bbf2b40eee2 not found: ID does not exist"
Aug 04 10:06:36.598336 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.598307    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"]
Aug 04 10:06:36.604343 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:36.604319    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-497679-64694cc976-tppzf"]
Aug 04 10:06:37.295215 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:37.295183    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a78d9c91-20d3-4b2b-b9de-1e522923266f" path="/var/lib/kubelet/pods/a78d9c91-20d3-4b2b-b9de-1e522923266f/volumes"
Aug 04 10:06:37.945304 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:37.945269    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:06:53.963118 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:53.963083    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"]
Aug 04 10:06:53.963693 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:53.963350    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp" podUID="2f53787b-8a2f-4c4f-9317-d709c7efbc5f" containerName="payload-processing" containerID="cri-o://0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7" gracePeriod=30
Aug 04 10:06:54.229327 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.229302    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 10:06:54.353939 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.353907    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vbhl\" (UniqueName: \"kubernetes.io/projected/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-kube-api-access-5vbhl\") pod \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") "
Aug 04 10:06:54.354095 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.353962    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-plugins-config-volume\") pod \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\" (UID: \"2f53787b-8a2f-4c4f-9317-d709c7efbc5f\") "
Aug 04 10:06:54.354331 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.354306    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f53787b-8a2f-4c4f-9317-d709c7efbc5f" (UID: "2f53787b-8a2f-4c4f-9317-d709c7efbc5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:06:54.356206 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.356184    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-kube-api-access-5vbhl" (OuterVolumeSpecName: "kube-api-access-5vbhl") pod "2f53787b-8a2f-4c4f-9317-d709c7efbc5f" (UID: "2f53787b-8a2f-4c4f-9317-d709c7efbc5f"). InnerVolumeSpecName "kube-api-access-5vbhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:06:54.455086 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.455054    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vbhl\" (UniqueName: \"kubernetes.io/projected/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-kube-api-access-5vbhl\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:54.455086 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.455079    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f53787b-8a2f-4c4f-9317-d709c7efbc5f-plugins-config-volume\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:06:54.667225 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.667195    2558 generic.go:356] "Generic (PLEG): container finished" podID="2f53787b-8a2f-4c4f-9317-d709c7efbc5f" containerID="0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7" exitCode=0
Aug 04 10:06:54.667392 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.667240    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"
Aug 04 10:06:54.667392 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.667237    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp" event={"ID":"2f53787b-8a2f-4c4f-9317-d709c7efbc5f","Type":"ContainerDied","Data":"0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7"}
Aug 04 10:06:54.667392 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.667364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp" event={"ID":"2f53787b-8a2f-4c4f-9317-d709c7efbc5f","Type":"ContainerDied","Data":"02bc25b09d3085426edeb1f11237652d897a4243857c4da63dc381dfbbbbb202"}
Aug 04 10:06:54.667392 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.667386    2558 scope.go:117] "RemoveContainer" containerID="0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7"
Aug 04 10:06:54.676294 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.676279    2558 scope.go:117] "RemoveContainer" containerID="0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7"
Aug 04 10:06:54.676569 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:06:54.676547    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7\": container with ID starting with 0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7 not found: ID does not exist" containerID="0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7"
Aug 04 10:06:54.676668 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.676575    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7"} err="failed to get container status \"0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7\": rpc error: code = NotFound desc = could not find container \"0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7\": container with ID starting with 0b73753807672d8f76d1ca11e38cb28653d79e57434f508f9596e176402e54a7 not found: ID does not exist"
Aug 04 10:06:54.689208 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.689180    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"]
Aug 04 10:06:54.693956 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:54.693937    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-b0bc81-67ff4946b6-bdxrp"]
Aug 04 10:06:55.296011 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:55.295966    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f53787b-8a2f-4c4f-9317-d709c7efbc5f" path="/var/lib/kubelet/pods/2f53787b-8a2f-4c4f-9317-d709c7efbc5f/volumes"
Aug 04 10:06:56.338761 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:06:56.338688    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-ksvrz"]
Aug 04 10:07:01.249952 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:01.249916    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"]
Aug 04 10:07:01.250386 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:01.250121    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" containerID="cri-o://a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded" gracePeriod=30
Aug 04 10:07:01.964422 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:01.964382    2558 patch_prober.go:28] interesting pod/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 10:07:01.964593 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:01.964442    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 10:07:06.510730 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.510697    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 10:07:06.659166 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659136    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-socket\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659187    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-istio-token\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659215    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8wxp\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-kube-api-access-m8wxp\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659243    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e55b90-84d2-418e-8718-817731535693-istiod-ca-cert\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659276    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-data\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659321 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659302    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-envoy\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659580 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659328    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-credential-socket\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659580 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659364    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-certs\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659580 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659386    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e55b90-84d2-418e-8718-817731535693-istio-podinfo\") pod \"d0e55b90-84d2-418e-8718-817731535693\" (UID: \"d0e55b90-84d2-418e-8718-817731535693\") "
Aug 04 10:07:06.659580 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659459    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:07:06.659839 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659677    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:07:06.659839 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659676    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e55b90-84d2-418e-8718-817731535693-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 10:07:06.659839 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659679    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.659839 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659817    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:07:06.660208 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.659850    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:07:06.661561 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.661537    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 10:07:06.661561 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.661544    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-kube-api-access-m8wxp" (OuterVolumeSpecName: "kube-api-access-m8wxp") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "kube-api-access-m8wxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:07:06.661692 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.661542    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 10:07:06.661752 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.661702    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0e55b90-84d2-418e-8718-817731535693-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0e55b90-84d2-418e-8718-817731535693" (UID: "d0e55b90-84d2-418e-8718-817731535693"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 10:07:06.726324 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.726283    2558 generic.go:356] "Generic (PLEG): container finished" podID="d0e55b90-84d2-418e-8718-817731535693" containerID="a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded" exitCode=0
Aug 04 10:07:06.726459 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.726331    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"
Aug 04 10:07:06.726459 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.726368    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" event={"ID":"d0e55b90-84d2-418e-8718-817731535693","Type":"ContainerDied","Data":"a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded"}
Aug 04 10:07:06.726549 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.726480    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8" event={"ID":"d0e55b90-84d2-418e-8718-817731535693","Type":"ContainerDied","Data":"31b849063cd826fff2c5c04abd8e6465ce2becbae9579aa833dc1aa185335a76"}
Aug 04 10:07:06.726549 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.726515    2558 scope.go:117] "RemoveContainer" containerID="a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded"
Aug 04 10:07:06.736623 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.736605    2558 scope.go:117] "RemoveContainer" containerID="efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5"
Aug 04 10:07:06.744666 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.744649    2558 scope.go:117] "RemoveContainer" containerID="a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded"
Aug 04 10:07:06.744944 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:07:06.744927    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded\": container with ID starting with a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded not found: ID does not exist" containerID="a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded"
Aug 04 10:07:06.745015 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.744953    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded"} err="failed to get container status \"a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded\": rpc error: code = NotFound desc = could not find container \"a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded\": container with ID starting with a694b56f34053281263b0ffc3818c4ea4e0bc7df05f4eeb839010b2300c56ded not found: ID does not exist"
Aug 04 10:07:06.745015 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.744971    2558 scope.go:117] "RemoveContainer" containerID="efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5"
Aug 04 10:07:06.745242 ip-10-0-143-106 kubenswrapper[2558]: E0804 10:07:06.745223    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5\": container with ID starting with efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5 not found: ID does not exist" containerID="efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5"
Aug 04 10:07:06.745289 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.745248    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5"} err="failed to get container status \"efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5\": rpc error: code = NotFound desc = could not find container \"efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5\": container with ID starting with efc6a4deb4626770ae1decfe645d80f514848a607a90c203e15c2f9a60dbe7a5 not found: ID does not exist"
Aug 04 10:07:06.752256 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.752234    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"]
Aug 04 10:07:06.757804 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.757784    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b0bc81-openshift-default-556678f9dd-prvw8"]
Aug 04 10:07:06.760358 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760340    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-istio-token\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.760413 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760362    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8wxp\" (UniqueName: \"kubernetes.io/projected/d0e55b90-84d2-418e-8718-817731535693-kube-api-access-m8wxp\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.760413 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760372    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e55b90-84d2-418e-8718-817731535693-istiod-ca-cert\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.760413 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760380    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-data\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.760413 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760388    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-istio-envoy\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.760413 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760395    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-credential-socket\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.760413 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760404    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e55b90-84d2-418e-8718-817731535693-workload-certs\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:06.760413 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:06.760412    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e55b90-84d2-418e-8718-817731535693-istio-podinfo\") on node \"ip-10-0-143-106.ec2.internal\" DevicePath \"\""
Aug 04 10:07:07.295504 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:07.295473    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e55b90-84d2-418e-8718-817731535693" path="/var/lib/kubelet/pods/d0e55b90-84d2-418e-8718-817731535693/volumes"
Aug 04 10:07:07.846766 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:07.846719    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-fb65c789d-rvt85_f61f28fc-b699-4998-8869-f42dbdc6ce88/manager/0.log"
Aug 04 10:07:08.445243 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:08.445216    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-plpvv_afc0c3dc-f1d0-4f57-9765-b8e750bc8b42/copy-manifests/0.log"
Aug 04 10:07:08.461319 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:08.461299    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-plpvv_afc0c3dc-f1d0-4f57-9765-b8e750bc8b42/manager/0.log"
Aug 04 10:07:09.189548 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:09.189518    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29763960-xsp8q_9e613e08-5e9d-4bbe-8679-0a81ece1e3f1/cleanup/0.log"
Aug 04 10:07:09.302542 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:09.302507    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-l9dq8_a2f1a578-5c90-4546-a28a-2a73660378ea/postgres/0.log"
Aug 04 10:07:10.233823 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:10.233795    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-8sqfh_ddaa39b9-0a08-4b34-bc82-a576086cf983/manager/0.log"
Aug 04 10:07:10.348801 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:10.348766    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-wbctw_89a290fa-fa2e-439f-a91c-03e06b0074c2/manager/0.log"
Aug 04 10:07:10.699149 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:10.699118    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-ksvrz_22cd4245-7cef-4ff7-be23-42d6099d2a61/limitador/0.log"
Aug 04 10:07:10.814577 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:10.814551    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-zgsbv_98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba/manager/0.log"
Aug 04 10:07:11.298853 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:11.298819    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rjr8s_a1442d06-baac-4b71-8db0-def11ce5f8ae/discovery/0.log"
Aug 04 10:07:11.407134 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:11.407111    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-9rjg7_07143299-3a85-4155-ab9d-5c3998986c39/kube-auth-proxy/0.log"
Aug 04 10:07:11.638232 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:11.638203    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b96pj_a4e8f32d-06b7-4a1a-b864-4661b8c330d6/istio-proxy/1.log"
Aug 04 10:07:11.864836 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:11.864809    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-kbzjm_d21cb922-e090-4055-b1a7-371acf923f6d/payload-processing/0.log"
Aug 04 10:07:12.463961 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:12.463924    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nznrm_d9d34197-686b-417e-8685-f39fc6cc5115/main/0.log"
Aug 04 10:07:12.947717 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:12.947680    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-ndmg4_ca2b29b1-689d-4d42-8a11-c9d9b1717728/main/0.log"
Aug 04 10:07:21.828584 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:21.828552    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kskw5_e4df6d71-b2d6-4b8d-af54-36717c5c4da5/global-pull-secret-syncer/0.log"
Aug 04 10:07:21.927050 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:21.927021    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-42hvz_b76a8da3-d1b6-472d-bdea-aea40ac4f05d/konnectivity-agent/0.log"
Aug 04 10:07:22.029136 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:22.029102    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-106.ec2.internal_862d5715eae1ca6077f67d5d3e2e5c27/haproxy/0.log"
Aug 04 10:07:26.805680 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:26.805652    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-8sqfh_ddaa39b9-0a08-4b34-bc82-a576086cf983/manager/0.log"
Aug 04 10:07:26.850830 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:26.850806    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-wbctw_89a290fa-fa2e-439f-a91c-03e06b0074c2/manager/0.log"
Aug 04 10:07:27.025553 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:27.025528    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-ksvrz_22cd4245-7cef-4ff7-be23-42d6099d2a61/limitador/0.log"
Aug 04 10:07:27.104771 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:27.104726    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-zgsbv_98c7f699-5bf8-4a8f-abc2-9b2b0dfcfaba/manager/0.log"
Aug 04 10:07:28.958116 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:28.958078    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f8f695f98-pcpbc_50757688-799c-4ce8-9781-f454b95b74c7/metrics-server/0.log"
Aug 04 10:07:29.028499 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.028462    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6jvw5_a409abeb-07c9-43f4-af5e-f9590d532649/node-exporter/0.log"
Aug 04 10:07:29.056668 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.056643    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6jvw5_a409abeb-07c9-43f4-af5e-f9590d532649/kube-rbac-proxy/0.log"
Aug 04 10:07:29.089403 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.089384    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6jvw5_a409abeb-07c9-43f4-af5e-f9590d532649/init-textfile/0.log"
Aug 04 10:07:29.459433 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.459335    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60353782-3641-4f05-a724-d6549148d5a5/prometheus/0.log"
Aug 04 10:07:29.481255 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.481209    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60353782-3641-4f05-a724-d6549148d5a5/config-reloader/0.log"
Aug 04 10:07:29.505084 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.505062    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60353782-3641-4f05-a724-d6549148d5a5/thanos-sidecar/0.log"
Aug 04 10:07:29.528608 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.528588    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60353782-3641-4f05-a724-d6549148d5a5/kube-rbac-proxy-web/0.log"
Aug 04 10:07:29.555055 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.555017    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60353782-3641-4f05-a724-d6549148d5a5/kube-rbac-proxy/0.log"
Aug 04 10:07:29.583718 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.583698    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60353782-3641-4f05-a724-d6549148d5a5/kube-rbac-proxy-thanos/0.log"
Aug 04 10:07:29.628215 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.628194    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60353782-3641-4f05-a724-d6549148d5a5/init-config-reloader/0.log"
Aug 04 10:07:29.666977 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.666955    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-gvqkx_873d73b6-c0cd-4f42-8af3-cbfdf390500c/prometheus-operator/0.log"
Aug 04 10:07:29.689600 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:29.689583    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-gvqkx_873d73b6-c0cd-4f42-8af3-cbfdf390500c/kube-rbac-proxy/0.log"
Aug 04 10:07:31.534601 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534565    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"]
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534938    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78d9c91-20d3-4b2b-b9de-1e522923266f" containerName="payload-pre-processing"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534951    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78d9c91-20d3-4b2b-b9de-1e522923266f" containerName="payload-pre-processing"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534965    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f53787b-8a2f-4c4f-9317-d709c7efbc5f" containerName="payload-processing"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534971    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f53787b-8a2f-4c4f-9317-d709c7efbc5f" containerName="payload-processing"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534980    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534985    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.534995    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535000    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535006    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3589357-c163-4c2f-9181-0e603b5229e8" containerName="revoke-keys"
Aug 04 10:07:31.535008 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535011    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3589357-c163-4c2f-9181-0e603b5229e8" containerName="revoke-keys"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535018    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535023    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535029    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535034    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535079    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535086    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f53787b-8a2f-4c4f-9317-d709c7efbc5f" containerName="payload-processing"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535094    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3589357-c163-4c2f-9181-0e603b5229e8" containerName="revoke-keys"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535100    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e55b90-84d2-418e-8718-817731535693" containerName="istio-proxy"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535108    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535114    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78d9c91-20d3-4b2b-b9de-1e522923266f" containerName="payload-pre-processing"
Aug 04 10:07:31.535315 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.535120    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="18b5f11c-4f3e-4475-bc21-6f4d489cee00" containerName="istio-proxy"
Aug 04 10:07:31.539751 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.539720    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.543294 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.543275    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddhlv\"/\"kube-root-ca.crt\""
Aug 04 10:07:31.544228 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.544208    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddhlv\"/\"openshift-service-ca.crt\""
Aug 04 10:07:31.544316 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.544208    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ddhlv\"/\"default-dockercfg-lb999\""
Aug 04 10:07:31.548758 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.548717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"]
Aug 04 10:07:31.581850 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.581824    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-sys\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.581968 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.581859    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-lib-modules\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.581968 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.581883    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-proc\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.581968 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.581901    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-podres\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.582100 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.582016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stq9f\" (UniqueName: \"kubernetes.io/projected/2df86464-2854-4b6f-8007-26063ce1446b-kube-api-access-stq9f\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683035 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.682995    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stq9f\" (UniqueName: \"kubernetes.io/projected/2df86464-2854-4b6f-8007-26063ce1446b-kube-api-access-stq9f\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683187 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683070    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-sys\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683187 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683100    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-lib-modules\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683187 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-proc\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683187 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683145    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-podres\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683354 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-sys\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683354 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-proc\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683354 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683251    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-podres\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.683354 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.683255    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2df86464-2854-4b6f-8007-26063ce1446b-lib-modules\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.703652 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.703621    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stq9f\" (UniqueName: \"kubernetes.io/projected/2df86464-2854-4b6f-8007-26063ce1446b-kube-api-access-stq9f\") pod \"perf-node-gather-daemonset-kj8rp\" (UID: \"2df86464-2854-4b6f-8007-26063ce1446b\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:31.850108 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:31.850081    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:32.000459 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:32.000432    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"]
Aug 04 10:07:32.003054 ip-10-0-143-106 kubenswrapper[2558]: W0804 10:07:32.003020    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2df86464_2854_4b6f_8007_26063ce1446b.slice/crio-4740f33e21389a24567b6cf5e0fed227740d44f072adec48ae7d10f895b43212 WatchSource:0}: Error finding container 4740f33e21389a24567b6cf5e0fed227740d44f072adec48ae7d10f895b43212: Status 404 returned error can't find the container with id 4740f33e21389a24567b6cf5e0fed227740d44f072adec48ae7d10f895b43212
Aug 04 10:07:32.554486 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:32.554399    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-f5k8d_7fc31d53-4275-4f0a-a52a-a6ddf8161869/download-server/0.log"
Aug 04 10:07:32.847369 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:32.847334    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp" event={"ID":"2df86464-2854-4b6f-8007-26063ce1446b","Type":"ContainerStarted","Data":"4630f96a2605e0ccea9ed07afb393ed5209cfd5d51282add4345561080f82edf"}
Aug 04 10:07:32.847536 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:32.847391    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp" event={"ID":"2df86464-2854-4b6f-8007-26063ce1446b","Type":"ContainerStarted","Data":"4740f33e21389a24567b6cf5e0fed227740d44f072adec48ae7d10f895b43212"}
Aug 04 10:07:32.847536 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:32.847418    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:32.865340 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:32.865293    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp" podStartSLOduration=1.865275866 podStartE2EDuration="1.865275866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:07:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:07:32.86447433 +0000 UTC m=+3584.122431382" watchObservedRunningTime="2026-08-04 10:07:32.865275866 +0000 UTC m=+3584.123232920"
Aug 04 10:07:34.320098 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:34.320072    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ds5r_c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f/dns/0.log"
Aug 04 10:07:34.344292 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:34.344274    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ds5r_c5bae8b5-8e93-46c8-9af4-f180ce3b8c5f/kube-rbac-proxy/0.log"
Aug 04 10:07:34.477017 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:34.476995    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dsh54_bcac14e8-3c2b-4665-a56e-7fc16b48043f/dns-node-resolver/0.log"
Aug 04 10:07:35.089769 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:35.089722    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8497dfc489-xnv7h_1fee394f-ab08-4450-8ed2-bae1f697907c/registry/0.log"
Aug 04 10:07:35.140459 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:35.140429    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pd75f_4ccbe261-c0cc-4deb-8575-27d331f415d3/node-ca/0.log"
Aug 04 10:07:36.257091 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:36.257062    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rjr8s_a1442d06-baac-4b71-8db0-def11ce5f8ae/discovery/0.log"
Aug 04 10:07:36.280192 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:36.280166    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-9rjg7_07143299-3a85-4155-ab9d-5c3998986c39/kube-auth-proxy/0.log"
Aug 04 10:07:36.339926 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:36.339899    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b96pj_a4e8f32d-06b7-4a1a-b864-4661b8c330d6/istio-proxy/0.log"
Aug 04 10:07:36.346220 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:36.346201    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b96pj_a4e8f32d-06b7-4a1a-b864-4661b8c330d6/istio-proxy/1.log"
Aug 04 10:07:36.409817 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:36.409792    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-kbzjm_d21cb922-e090-4055-b1a7-371acf923f6d/payload-processing/0.log"
Aug 04 10:07:37.278114 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:37.278084    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t9sl8_47752d1b-8f63-439d-b354-532f3b7ed228/serve-healthcheck-canary/0.log"
Aug 04 10:07:37.966634 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:37.966605    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k5hsh_646ef199-644e-4b1c-83c6-8367c369dd61/kube-rbac-proxy/0.log"
Aug 04 10:07:37.988294 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:37.988275    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k5hsh_646ef199-644e-4b1c-83c6-8367c369dd61/exporter/0.log"
Aug 04 10:07:38.012588 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:38.012561    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k5hsh_646ef199-644e-4b1c-83c6-8367c369dd61/extractor/0.log"
Aug 04 10:07:38.862084 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:38.862058    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-kj8rp"
Aug 04 10:07:40.905889 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:40.905858    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-fb65c789d-rvt85_f61f28fc-b699-4998-8869-f42dbdc6ce88/manager/0.log"
Aug 04 10:07:41.268659 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:41.268623    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-plpvv_afc0c3dc-f1d0-4f57-9765-b8e750bc8b42/manager/0.log"
Aug 04 10:07:41.288896 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:41.288875    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-plpvv_afc0c3dc-f1d0-4f57-9765-b8e750bc8b42/copy-manifests/0.log"
Aug 04 10:07:42.285951 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:42.285919    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rjr8s_a1442d06-baac-4b71-8db0-def11ce5f8ae/discovery/0.log"
Aug 04 10:07:42.297827 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:42.297798    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-9rjg7_07143299-3a85-4155-ab9d-5c3998986c39/kube-auth-proxy/0.log"
Aug 04 10:07:42.346475 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:42.346444    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b96pj_a4e8f32d-06b7-4a1a-b864-4661b8c330d6/istio-proxy/0.log"
Aug 04 10:07:42.352255 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:42.352236    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b96pj_a4e8f32d-06b7-4a1a-b864-4661b8c330d6/istio-proxy/1.log"
Aug 04 10:07:42.399193 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:42.399159    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-kbzjm_d21cb922-e090-4055-b1a7-371acf923f6d/payload-processing/0.log"
Aug 04 10:07:44.093846 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:44.093818    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rjr8s_a1442d06-baac-4b71-8db0-def11ce5f8ae/discovery/0.log"
Aug 04 10:07:44.105520 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:44.105484    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-9rjg7_07143299-3a85-4155-ab9d-5c3998986c39/kube-auth-proxy/0.log"
Aug 04 10:07:44.159110 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:44.159086    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b96pj_a4e8f32d-06b7-4a1a-b864-4661b8c330d6/istio-proxy/0.log"
Aug 04 10:07:44.165114 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:44.165088    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-b96pj_a4e8f32d-06b7-4a1a-b864-4661b8c330d6/istio-proxy/1.log"
Aug 04 10:07:44.214707 ip-10-0-143-106 kubenswrapper[2558]: I0804 10:07:44.214657    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-kbzjm_d21cb922-e090-4055-b1a7-371acf923f6d/payload-processing/0.log"