Artykuł w czasopiśmie
Brak miniatury
Licencja
Dostęp zamkniętyNon-asymptotic Analysis of Biased Stochastic Approximation Scheme
| dc.abstract.en | Stochastic approximation (SA) is a key method used in statistical learning. Recently, its non-asymptotic convergence analysis has been considered in many papers. However, most of the prioranalyses are made under restrictive assumptions such as unbiased gradient estimates and convexobjective function, which significantly limit their applications to sophisticated tasks such as onlineand reinforcement learning. These restrictions are all essentially relaxed in this work. In particular,we analyze a general SA scheme to minimize a non-convex, smooth objective function. We con-sider update procedure whose drift term depends on a state-dependent Markov chain and the meanfield is not necessarily of gradient type, covering approximate second-order method and allowingasymptotic bias for the one-step updates. We illustrate these settings with the online EM algorithmand the policy-gradient method for average reward maximization in reinforcement learning.Keywords:biased stochastic approximation, state-dependent Markov chain, non-convex optimiza-tion, policy gradient, online expectation-maximization. |
| dc.affiliation | Uniwersytet Warszawski |
| dc.conference.country | Stany Zjednoczone |
| dc.conference.datefinish | 2019-06-28 |
| dc.conference.datestart | 2019-06-25 |
| dc.conference.place | Phoenix |
| dc.conference.series | Conference on Learning Theory |
| dc.conference.series | Conference on Learning Theory |
| dc.conference.seriesshortcut | COLT |
| dc.contributor.author | Miasojedow, Błażej |
| dc.contributor.author | Wai, Hoi-To |
| dc.contributor.author | Moulines, Eric |
| dc.contributor.author | Karimi, Belhal |
| dc.date.accessioned | 2024-01-25T13:50:40Z |
| dc.date.available | 2024-01-25T13:50:40Z |
| dc.date.issued | 2019 |
| dc.description.finance | Nie dotyczy |
| dc.description.volume | 99 |
| dc.identifier.issn | 2640-3498 |
| dc.identifier.uri | https://repozytorium.uw.edu.pl//handle/item/113824 |
| dc.identifier.weblink | http://proceedings.mlr.press/v99/karimi19a.html |
| dc.language | eng |
| dc.pbn.affiliation | mathemathics |
| dc.relation.ispartof | Proceedings of Machine Learning Research |
| dc.relation.pages | 1944-1974 |
| dc.rights | ClosedAccess |
| dc.sciencecloud | nosend |
| dc.subject.en | biased stochastic approximation |
| dc.subject.en | state-dependent Markov chain |
| dc.subject.en | non-convex optimiza- tion |
| dc.subject.en | reinforcement learning |
| dc.subject.en | online expectation-maximization |
| dc.title | Non-asymptotic Analysis of Biased Stochastic Approximation Scheme |
| dc.type | JournalArticle |
| dspace.entity.type | Publication |