An input generalization problem is one of the most important ones in applying reinforcement learning to real robot tasks. To cope with this problem, we propose a self-partitioning state space algorithm which can make non-uniform quantization of the multidimensional continuous state space. This method recursively splits its continuous state space into some coarse spaces called tentative states. It begins by supposing that such tentative states are regarded as the states for Q-learning. It collects Q values and statistical evidence regarding immediate rewards r and Q values within this tentative state space. When it finds out that a tentative state is relevant by the statistical test on minimum description length criterion, it partitions this coarse space into finer spaces. These procedures can make non-uniform quantization of the state space. Our method can be applied to non-deterministic domain because Q-learning is used to find out the optimal policy for accomplishing the given task. To show that our algorithm has generalization capability, we apply our method to two tasks in which a soccer robot shoots a ball into a goal and prevent a ball from entering a goal. To show the validity of this method, the experimental results for computer simulation and a real robot are shown.
['Takayuki Nakamura']
Development of Self-Learning Vision-Based Mobile Robots for Acquiring Soccer Robots Behaviors