Tomás Martínez-Marín and Tom Duckett

Fast Reinforcement Learning for Vision-guided Mobile Robots

Proc. ICRA-2005, IEEE International Conference on Robotics and Automation


Abstract

This paper presents a new reinforcement learning algorithm for accelerating acquisition of new skills by real mobile robots, without requiring simulation. It speeds up Q-learning by applying memory-based sweeping and enforcing the ``adjoining property'', a technique that exploits the natural ordering of sensory state spaces in many robotic applications by only allowing transitions between neighbouring states. The algorithm is tested within an image-based visual servoing framework on a docking task, in which the robot has to position its gripper at a desired configuration relative to an object on a table. In experiments, we compare the performance of the new algorithm with a hand-designed linear controller and a scheme using the linear controller as a bias to further accelerate the learning. By analysis of the controllability and docking time, we show that the biased learner could improve on the performance of the linear controller, while requiring substantially lower training time than unbiased learning (less than 1 hour on the real robot).

Download

pdf (http)

pdf (ftp)

Bibtex 

@INPROCEEDINGS{martinez05,
  TITLE = {Fast Reinforcement Learning for Vision-guided Mobile Robots},
  AUTHOR = {Mart\'{i}nez-Mar\'{i}n, Tom\'{a}s  and Duckett, Tom},
  BOOKTITLE = {IEEE International Conference on Robotics and Automation (ICRA 2005)},
  ADDRESS = {Barcelona, Spain},
  YEAR = {2005},
  MONTH = {April 18-22}
}