Survey of Model-Based Reinforcement Learning: Applications on Robotics
Abstract Reinforcement learning is an appealing approach for allowing robots to learn new tasks. Rel- evant literature reveals a plethora of methods, but at the same time makes clear the lack of implementations for dealing with real life challenges. Current expecta- tions raise the demand for adaptable robots. We argue that, by employing model-based reinforcement learn- ing, the—now limited—adaptability characteristics of robotic systems can be expanded. Also, model-based reinforcement learning exhibits advantages that makes it more applicable to real life use-cases compared to model-free methods. Thus, in this survey, model- based methods that have been applied in robotics are covered. We categorize them based on the deriva- tion of an optimal policy, the definition of the returns function, the type of the transition model and the learned task. Finally, we discuss the applicability of model-based reinforcement learning approaches in new applications, taking into consideration the state of the art in both algorithms and hardware.
摘要强化学习是一种允许机器人学习新任务的吸引人的方法。相关文献揭示了许多方法，但同时也清楚地表明，缺乏应对现实生活挑战的实现方法。当前的期望提高了对自适应机器人的需求。我们认为，通过采用基于模型的强化学习，可以扩展机器人系统的（现在有限的）适应性特征。此外，与基于模型的强化学习相比，基于模型的强化学习具有优势，与无模型方法相比，它更适用于现实生活中的用例。因此，在本次调查中，涵盖了已在机器人技术中应用的基于模型的方法。我们根据最佳策略的派生，收益函数的定义，过渡模型的类型和学习到的任务对它们进行分类。最后，考虑到算法和硬件方面的最新技术，我们讨论了基于模型的强化学习方法在新应用程序中的适用性。