2052410
Reinforcement Learning for CPG-Driven Biped Robot
Animal's rhythmic movements such as locomotion are considered to be controlled by neural circuits called central pattern generators (CPGs). This article presents a reinforcement learning (RL) method for a CPG controller, which is inspired by the control mechanism of animals. Because the CPG controller is an instance of recurrent neural networks, a naive application of RL involves difficulties. In addition, since state and action spaces of controlled systems are very large in real problems such as robot control, the learning of the value function is also difficult. In this study, we propose a learning scheme for a CPG controller called a CPG-actor-critic model, whose learning algorithm is based on a policy gradient method. We apply our RL method to autonomous acquisition of biped locomotion by a biped robot simulator. Computer simulations show our method is able to train a CPG controller such that the learning process is stable.
26916794
Natural Policy Gradient Reinforcement Learning for a CPG Control of a Biped Robot
Motivated by the perspective that animals’ rhythmic movements such as locomotion are controlled by neural circuits called central pattern generators (CPGs), motor control mechanisms by CPG have been studied. As an autonomous learning framework for a CPG controller, we previously proposed a reinforcement learning (RL) method called the CPG-actor-critic method. In this article, we propose a natural policy gradient learning algorithm for the CPG-actor-critic method, and applied our RL to an automatic control problem by a biped robot simulator. Computer simulations show that our RL makes the biped robot walk stably on various terrain.
798006
Efficient parallel dynamics computation of human figures
An efficient parallel algorithm for forward dynamics computation of human figures is proposed. The algorithm is capable of handling any kinematic chains including structure-varying ones. The asymptotic complexity of the algorithm is O(N) in serial computation and O(log N) in parallel computation on O(N) processors for most practical kinematic chains. The idea is to assemble a kinematic chain by adding the joints one by one and compute the constraint forces at the new joints using the principle of virtual work. The parallelism of the algorithm can be adapted for parallel processing systems with any number of processors by simply changing the assembly order. Simulation examples on an 8-node cluster demonstrate the effectiveness of the algorithm.